More or less, yes.
That is: apart from the colourful details like "shortly before his death", and 'doing differently if living that live again' — for both of which there is no evidence I could find. But this following sentence sums up the state of affairs:
Albert Einstein (1879–1955) studied some Talmud as a youth, supported Talmud study and regretted not studying more.
— David E. Y. Sarna: "Studying Talmud: The Good, the Not-So-Good and How to Make Talmud More Accessible", JewishLink, January 19, 2017.
These three data-points are widely quoted in his biographies, like this:
This attitude is evidenced in his replies to some questions raised by George Sylvester Viereck during a 1929 interview:
“To what extent are you influenced by Christianity?”
“As a child I received instruction both in the Bible and in the Talmud. I am a Jew, but I am enthralled by the luminous figure of the Nazarene.”
[…]
From 1892 to 1895, the year Albert left Munich to join his parents in Italy without having completed his schooling, his teachers of religion were Dr. Joseph Perles, Eugene Meyer, and Dr. Cossmann Werner. They introduced him to the literature of the Psalms, and the history of the Talmud and of the Jews in Spain. Unfortunately, because these external teachers did not enjoy the same authority as their full-time colleagues at the Gymnasium, the attitude of their pupils toward their lessons seems to have been less serious that it should have been. Einstein referred to this in 1929 when he received fiftieth-birthday congratulations from his old teacher Heinrich Friedmann. Einstein declared:
“I was deeply moved and delighted by your congratulations. How vividly do I remember those days of my youth in Munich and how deeply do I regret not having been more diligent in studying the language and literature of our fathers. I read the Bible quite often, but the original text remains inaccessible for me. It certainly was not your fault; you have fought valiantly and energetically against laziness and all kinds of naughtiness.”
[…]
— Max Jammer: "Einstein and Religion. Physics and Theology", Princeton University Press: Princeton, 1999.
And the promotion of the more widespread study of this kind of literature is evidenced by:
The scientific organization and comprehensive exposition in accessible form of the Talmud has a twofold importance for us Jews. It is important in the first place that the high cultural values of the Talmud should not be lost to modern minds among the Jewish people nor to science, but should operate further as a living force. In the second place, The Talmud must be made an open book to the world, in order to cut the ground from under certain malevolent attacks, of anti-Semitic origin, which borrow countenance from the obscurity and inaccessibility of certain passages in the Talmud. To support this cultural work would thus mean an important achievement for the Jewish people.
— From a letter by Albert Einstein to Professor Chaim Tchernowitz (31 December 1930) of the Jewish Institute of Religion in New York (Hebrew Union College). Jewish Telegraphic Agency (Jewish Daily Bulletin)
It is important to note that the above might lend itself to over-interpret this in 'religious terms', when it really only is proof for a certain cultural affiliation, which the following quotes should orient into a more balanced direction:
Second, it is absurd to claim that Einstein was influenced by the Talmud. Although he once declared “that as a child he received instruction in the Bible and in the Talmud,” there can be no doubt that he never really studied the Talmud; for German Jews, unlike the Jews of Eastern Europe, rarely read the Talmud. It might perhaps be objected that Einstein had been indirectly influenced by the Talmud through Spinoza, because the young Spinoza had studied the Talmud in Amsterdam when he was a disciple of Rabbi Manasse BenIsrael whose portrait has been immortalized by Rembrandt. It was not the Talmudist from Amsterdam, but the philosopher of Voorburg, the author of the Ethics whom Einstein admired and with whom he felt a kinship.
[…]
Rabbi Hyman Cohen of the West New York and Guttenberg Talmud Torah commented: “Einstein is emphatically no atheist. He believes in a God. But in renouncing a personal deity he removes the Supreme Being so remotely from the sphere of human comprehension as to make His influence on the individual's conduct negligible. To use mathematical terminology, he reduces the infinite to an infinitesimal of the highest order…. Einstein is unquestionably a great scientist, but his religious views are diametrically opposed to Judaism.”
— Jammer
