The factory owners certainly did not start the habits of English workers to drink their tea with milk to speed up the breaks.
That is a bogus claim incompatible with recorded history. Perhaps there were a few factory owners that made the observation that tea cooling faster might reduce time spent on breaks and thus wanted their workers to add milk in any case, even if they'd prefer it without. But milk or cream was already en vogue before.
Unless a source appears in which any one factory owner reasons along these lines in a letter or diary, this lesser claim of 'owners did not start but promoted adding milk for speed reasons' should be also not taken overly seriously.
Milk in tea is a habit that is older than the industrial revolution, started in Britain with the upper classes and slowly filtered down.
— William Hogarth "The Strode Family", c.1738 (Notice the little cream jug in the centre of the table) via Tate
It came to be popular mainly probably because of simple taste. English tea was and often still is of so low a quality — or plainly bitter — that milk takes off a large part of this 'edge'.
Reasons for the bitterness now are preference for strongness, cheap bulk varieties, and for green and the still more popular black varieties a generally 'too hot' brewing process and sometimes even too long extraction process. Not in the least this was also caused by the slow speed of delivery to Europe before the adoption of clipper type sailing vessels and the 'invention' of selling even fannings swept up from factory floors.
That British factory owners invented such a habit is also quite a curious claim if we look at other societies and their preferences for tea with milk, be it in China, India, Tibet, Mongolia, Russia (in this case: not), Taiwan, Burma, Turkey.
Apart from quite old and significant Frisian habits to always include some dairy, tea arrived in Western Europe mainly through the Dutch at first.
In 1610 the first shipment of Chinese tea reached The Hague, and wealthy patrons were dazzled by it. The Dutch embraced tea with a fervent passion, and they laced it heavily with milk based on reports from Dutch traders that this was how the Chinese emperor took his tea. Because the emperor at the time was the Manchu emperor, these reports were based on information that was only true for him; Han Chinese emperors never did nor never would add milk to their tea.
After the Dutch adopted the tea habit, members of the French upper class began to drink tea as well. In Paris the Marquise de Sevigné, a cultured woman of letters, extolled the way that her friend Mme. de la Sablière drank “tea à la Chinoise” (or tea with milk). Tea reached Germany about 1650, and was first mentioned to have appeared in Scandinavia in 1723. But it was not until 1658 that the first public sale of Dutch-traded Chinese tea commenced in London at Garraway’s Coffee House.
When in 1662 Charles II wed Princess Catherine of Braganza, a Portuguese princess and tea drinker, tea became the fashionable beverage for English ladies. This opened the way for the rapid rise of the social traditions of “teatime.” Like the Dutch, the English added milk to their tea. They also added lumps of sugar, which England imported in vast quantities from the West Indies. Sugar added another boost to the energizing effects of tea, and a cup of black tea with cream and sugar defined the English style of twice-daily tea drinking.
— Mary Lou Heiss & Robert J. Heiss: "The story of tea: a cultural history and drinking guide", Ten Speed Press: New York, 2007.
(For national preferences, see — Helen Saberi: "Tea A Global History", Edible Series, Reaktion Books: London, 2010. From that book the following quote:)
In 1657 Thomas Garraway opened his coffee house in London and extolled the virtues of tea as being ‘quite refined, which could be presented to princes and other great people’. Garraway also claimed that tea ‘being prepared and drunk with milk and water it strengtheneth the inward parts’ which suggests that in England milk was sometimes added to tea right from the beginning. Another early reference to milk in tea came from France via the pen of Madame de Sévigné. In a letter written in 1680 to a friend who was in poor health she advised her to drink milk and recommended that to avoid the cold milk clashing with the heat of the blood she should to add it to hot tea. She added that Mme de la Sablière recently took ‘tea with her milk’ because she liked it.
There is much debate about adding milk in tea and why this tradition started in the West. It has been suggested that milk was added to tea to prevent cracking delicate porcelain cups. Another question which is still hotly debated is whether milk should be added first to the cup or last. However, milk was not a common addition to tea in England before the 1720s. It was about this time that black tea overtook green tea in popularity and this could have played a part. Milk may have been added to offset the bitterness of the tea. Sugar was also added for the same reason.
The following excerpts should cover most of the bases for the claim:
‘Lacte et carne vivant’ (they live on milk and meat) wrote Julius Caesar of the British.
The promotion of milk began in 1922 with the formation of the National Milk Publicity Council, adopting the slogan ‘Drink More Milk’ two years later in a campaign to persuade the public that milk was a healthy and nutritious drink, equally good for adults and children, and targeting outdoor and sporting activities.
The precise date of tea’s appearance in England is disputed between 1591 and 1612, the latter probably the more likely; at the phenomenal price of £6 10s a pound, the supply came from Holland, where the Dutch East India Company had opened commercial relations with China ahead of its English rival. It was still little known in the first half of the seventeenth century, when coffee and chocolate were already making progress, but its acceptance in aristocratic circles was encouraged by the Restoration of Charles II in 1660 when his Portuguese Queen, Catherine of Braganza, brought her previously acquired taste to the English Court. From the first, tea had strong gender associations. Although it was served in some of the exclusively male coffee-houses that opened in London after 1652 (see Chapter 4) it was not usual in these before the 1690s, and was never the principal drink there. While coffee-drinking began among men in public, the consumption of tea was primarily domestic, beginning in wealthy households where its service was associated with other novel objects of conspicuous display — fine china porcelain teapots, cups and saucers, gilded mahogany tea-tables and matching chairs (as at Ham House in 1683) and silver tea equipages consisting of teapot, tea-kettle, milk or cream jug, sugar bowl and spoon-tray.
In 1700 tea was still an occasional drink of the wealthy, fashionable few: before the end of the century it was regularly consumed by all social classes, and formed an integral part of the new dietary patterns of the poor. This promotion of tea from restricted to mass consumption was a process rather than an event, and cannot be dated precisely, but when in 1784 William Pitt slashed the customs duty on tea from 119 per cent to 121⁄2 per cent it was a recognition that it was now a normal beverage of the British people, worthy of encouragement, for
Tea has become an economical substitute in the middle and lower classes of society for malt liquor, the price of which renders it impossible for them to procure the quantity sufficient for them as their only drink.
Two other reasons are more important in accounting for the central place that tea came to occupy in working-class diets. Sugar early became associated with the use of tea, adding sweetness, palatability and energy in the form of calories to a drink that otherwise lacked nutritional value. When added sugar became usual is uncertain, but probably within the first two or three decades of the eighteenth century, at the time when black Bohea and Congou teas, stronger and more bitter than green, were becoming popular, especially with working-class consumers.
By this time tea or, less commonly, coffee, had generally replaced beer for breakfast; if resources allowed, men drank beer with their supper, but otherwise tea was the family beverage, drunk with and between meals through the day. Arthur Young in 1767 complained of ‘men making tea an article of their food almost as much as women, labourers losing their time to go and come to the tea table’, while in 1797 Eden observed that labourers’ families in Middlesex and Surrey drank tea three times a day with their meals.
The poorest labourers are habituated to the unvarying meal of dry bread and cheese, and from week to week’s end in these families whose finances do not allow them the indulgence of malt liquor, the deleterious product of China constitutes their most usual and general beverage.
How tea was made in such households at this time is uncertain, but in some it seems that the tea-kettle was kept simmering on the hob throughout the day, more leaves being added as required: in poor homes ‘spent’ leaves were dried and reused and ‘donkey tea’ made of burnt crusts was sometimes substituted. Eden noted that at this level tea was often drunk without either milk or sugar, and David Davies, one of the few sympathetic social observers to defend the poor’s use of tea, commented
Spring water, just coloured with the leaves of the lowest-priced tea and sweetened with the brownest sugar is the luxury for which you reprove them. To this they have recourse from mere necessity, and were they now to be deprived of this they would immediately be reduced to bread and water. Tea-drinking is not the cause, but the consequence of the distresses of the poor.
Although the adoption of tea by all classes now seemed irreversible, its place in the nation’s diet continued to be controversial. While Samuel Johnson, who described himself as ‘a hardened and shameless tea-drinker’, might be indulged, and the middle and upper classes who refreshed themselves with tea after substantial meals or took an evening’s recreation in a Tea Garden44 could even be admired for their sobriety, many still thought, like Hanway, that tea-drinking by the poor was ‘pernicious to Health, obstructing industry and impoverishing the Nation’. In one respect the critics were right: the new diet was less nutritious than the old, but it was the result of forces of economic change and ‘modernization’ rather than of choice or extravagance.
In 1863 Dr Edward Smith believed that tea was now a necessity, ‘not from the requirements of the body, but from the acquired habits and tastes of the people’; such families were now consuming half a pound a week, estimated to cost £5 4s a year, almost as much as they spent on meat (£6 18s 8d) and half the cost of rent. As well as consumption in the home, workers took tea to be brewed at factory breaks, agricultural labourers boiled a kettle in the fields, and miners took cans of cold tea into the pits. In middle-class homes tea was now more usual than coffee, and only in the wealthiest households did both appear as alternatives at breakfast and after dinner. The later hour of dining in Victorian England — often at 7.30 p.m. or 8 p.m. in the highest circles — created a gap for an additional light meal, ‘afternoon tea’, at around 4 p.m., a social occasion mainly for ladies and eligible bachelors. Although the English tea ceremony never developed the elaboration of the Japanese, the arts of managing the tea equipment and serving guests were marks of social accomplishment, as was the manner of drinking. Victorian etiquette required the milk or cream (tea-sets always included a ‘cream’ jug) to be added after the tea, allowing the drinker to decline or limit the amount: to drink from the saucer was no longer acceptable, while a teaspoon laid across the cup indicated that the drinker declined a refill. The polite ‘afternoon tea’, at which little more than bread and butter or sandwiches were served, was quite different from ‘high tea’, a substantial meal of cold meats or fish, salads, fruit and cakes that developed in the later nineteenth century, particularly in the north of England and Scotland.
From July 1940 tea was rationed at 2 oz a week for all over the age of 5, increased in 1943 to 3 oz for people over 70, and there were additional allowances for merchant seamen, harvesters, blast furnace workers and others. Lord Woolton, the Minister of Food, resisted the idea of a single blend of ‘pool’ tea, arguing that ‘Taste, individual taste, is worth preserving’: instead, tea was allocated to the companies in three grades, common, medium and fine, leaving it to the blenders to make the best use and enabling the principal firms to maintain some of their popular brands. Large quantities of tea were also drunk outside the home in canteens and British restaurants and at tea-breaks in factories, strongly encouraged by Ernest Bevin, the Minister of Labour, as an aid to productivity. Consumption per head remained throughout the war at around 9 lb a year, only marginally below that of the late 1930s, while the average price rose only from 2s 101⁄2d a pound in 1942 to 3s 1d in 1945.
— John Burnett: "Liquid Pleasures. A Social History of Drinks in Modern Britain", Routledge: London, New York, 1999.
An easier to check, because it's freely available online, version of a BA thesis surrounding the topic would be:
— Kendra H. Wilhelm: "A Different Cup of Tea: The Culture of Tea in Britain and Sri Lanka", Vassar College, 1994. Chapter "Tea comes to England", generously quoted in Ni Wang: "A Comparison of Chinese and British Tea Culture", Asian Culture and History Vol. 3, No. 2; July 2011. (doi: 10.5539/ach.v3n2p13 PDF)