There are developers, which move every static text on a webpage to custom labels, because they think, it is faster and it's good practice to have all text in custom labels.
Let's debunk this hoax.
From Salesforce Help-
"Custom labels enable developers to create multilingual applications by automatically presenting information (for example, help text or error messages) in a user’s native language. Custom labels are custom text values that can be accessed from Apex classes, Visualforce pages, Lightning pages, or Lightning components. The values can be translated into any language Salesforce supports."
From above note, we can convert text from English to any language salesforce supports. e.g. if you are designing Menu for a page and wants to automatically convert menu items into other language depending on your requirement, then custom labels would come in handy.
More importantly, custom labels are not cached. They are pulled when they are referenced.
You can create up to 5,000 custom labels for your organization, and they can be up to 1,000 characters in length. Custom labels from managed packages don’t count toward this limit.
Now, the real question, are custom labels render faster than static text on a page?
I don't think so, I never came across any article which states that custom labels render faster on page. Moreover, If you overuse custom labels, you will hit the limit eventually and have to limit yourself when you really need custom label.
So, in my opinion, don't go crazy and never move every small text to custom label.