Not that this claim over "belongs to" or "return" should matter that much for any argument in the present day, but of course this is rooted in historical fact, with a twist if you go back further still:
As suspected in the question:
"Putin usually doesn't go for complete historical fabrication, …"
The historical region of Ingria or Ingermanland is exactly what would match such a description. Ingria or Ingermanland is at the edge of the Baltic Sea, bordering the Finnish Gulf — and is roughly around the place in which Peter founded St Petersburg.
This area has changed hands quite a bit between for example Sweden, Teutonic Knights, Novgorod Republic, Muscowy, or after Ivan the Terrible, the Tsardom of Russia.
This area was indeed often under Novgorod, then Muscowy and then Russian influence, or control; or even genuine part of that state. Whether that makes it "belong" to either Sweden, Finland, to the Votians, Ingrians or Izhors, or Russians, at any time is up for grabs, and basing anything on history that far back is … you know… let's say bad style.
The biggest 'Russian' stronghold in that region would thus be perhaps Ivangorod:
The fortress, established in 1492 during the reign of Ivan III of Moscow took its name (literally: Ivan-town — gorod in Russian means "town" or "city") from that of the Tsar. Between 1581 and 1590 and from 1612 to 1704, Sweden controlled the area.
The near by and even older Schlüsselburg Fortess (after Peter renamed it in German, before: Oreshek (Орешек) or Orekhov (Орехов) shows a similar story. Only that the Russian side tells it would have been an originally Russian construction under Yuriy Danilovich of Moscow, while the Swedish sides claims, well, … say that Torkel Knutsson built in 1299… (which is true for Vyborg)
On German Wikipedia this illustration shows "Russian Reconquests" in roughly that area after the events that followed the Treaty of Nöteborg from 1323–1714:

A very nice historical sketch for the roundabout area is as follows:
In the 11th century, another East-Slavonic tribe reached the neighbourhood of the region inhabited by the northern Votians, where in the 13th and 14th centuries the former began to found their settlements in the lngrian Upland. That started the assimilation of Votians with Eastern Slavs. Purely Votian settlements were preserved only on the margins of orthodox faith, which now began to spread into the land of the Votians. During the crusades of the the Ingrians or Izhors. […]
Under the Stolbova peace treaty (1607) the greatest part of the land populated by Votians was incorporated into the Swedish- Finnish state. Some of the Votians adopted the Lutheran faith and fused with the Ingrian Finns, while others emigrated to regions inhabited by Russians, where they
became Russified. After the annexation of the land inhabited by Votians to Russia in 1704, large numbers of Russians were settled among Votians, along with the imposition of serfdom, which lasted until 1861. […]
Since ancient times Votian's neighbours have been the Ingrians or Izhors. They are descendents of the Karelians who settled on the banks of the Rivers Neva and Inkere/Izhora in the llth century. The name of the latter river has provided the Finnish (inkeroinen, pl. inkeroiset; inkerikko, pl. inkerikot) and Russian (i~ora) versions of the name for these people. Also the present Estonian ethnonym (isur, pl. isurid) is derived from Russian. The name of Voteland fell into disuse and the region came to be called Inkeri in Finnish, Ingermanland in Swedish and, at first, I~orskaia zemlia in Russian. In 1704 the region that had passed from Swedish to Russian rule was named Ingermanlandskaia guberniia, which in 1710 was changed to Sankt-Peterburgskaia guberniia. […]
The areas settled by Ingrians came under the influence of Novgorod as early as the llth and 12th centuries. Later they participated together with Russians in campaigns against Sweden. In 1300 the Swedes founded a stronghold called Landskrona (in Finnish Maankruunu) at the mouth of the River Neva in the land of Ingria, which, however, was destroyed the next year. The frontier drawn under the Ndteborg/Oregek peace treaty, concluded between Sweden and Novgorod in 1323,separated Ingrians from Karelians of Ayr~ipg~i(in the western part of the Karelian Isthmus) who came under Swedish rule. In this region,
inhabited by Ingrians, Russians built a stronghold called Oregek (Ndteborg in Swedish, P~ihkin~ilinna in Finnic languages). The uncertain situation in the border area compelled the Ingrians to move west (as far as the Rivers Narva and Lauga/Luga) and south (to the surroundings of the River Orede~). Already in the 16th and 17th centuries most of the people inhabiting the former land of Votians were Ingrians. The prevailing religion of the population was the Orthodox faith, especially from the year 1534. After Ingria's incorporation into Sweden the Ingrians adopted the Lutheran faith, while a segment of the Ingrians professing the Orthodox faith moved away to regions inhabited by Russians (Laanest 1964). […]
In 1570-1595 the Swedish-Russian War badly devastated Ingria. As a result, a large segment of its Orthodox population fled to areas inhabited by Russians. As to the part of Ingria that came under Swedish rule, a certain number of settlers professing the Lutheran faith moved there from Finland. As the Tgyssin/i peace treaty of 1595 had left Ingria to Russia, a new war started in 1610, which ended with the Stolbova peace treaty of 1617.Now Ingria - the former Russian districts Ivangorod, Iamburg, Koporie and Oregek - was incorporated into Sweden. This area stretched for 200 km W to E between the Rivers Narva and Lava, and, on an average, for 130 km from N to S from the Karelian border at Viborg to the Russian border. The land frontier ran for 110 km in the Karelian isthmus bordering on Old Finland (Viborg and Kexholm/K~ikisalmi Karelia) and in the S it stretched for 240 km, bordering on Russia. Apart from that, the boundary of Ingria extended for 210 km along the coast of the Gulf of Finland and for 110 km along the shore of Lake Ladoga. From that time onwards a new period of demographic development began on this territory of 15,000 km2.
The Stolbova peace treaty allowed the boyars, monks
and citizens to leave Orthodox Ingria for Russia. In actual fact, a large number of peasants of the Orthodox faith also left the region. The latter were sent by the Russian authorities to the villages that had lost their inhabitants as a result of the hostilities and famines. At first, Swedish authorities intended to resettle Ingria with German colonists, but by 1650 only 24 German families had moved into the area. Nor did the authorities succeed in settling Swedish peasants there. As there was no longer any state frontier in the Karelian isthmus, peasants from Savo/ Savolax (Savakot) and Viborg now moved into Ingria, especially into its northern and central parts. As can be seen from the public registers of landed property for 1618-1623, two-thirds of the people who moved over to the districts of Kaprio/Koporie and Iivananlinna/Ivangorod had come from the parish of Ayriip~i~i(Ayfiim6iset) in the Karelian isthmus. They had arrived in the past 40 years, some of them even earlier.
Another blow to the development of the settlements in Ingria was dealt by an invasion of Russian troops in 1656. Two years of hostilities decreased the population considerably. Again, part of the people left together with the Russian army, including Lutherans being taken along. By 1661, when the K~irde/Kardis peace treaty was concluded, only one-third of the pre-war population remained in Ingria. After the conclusion of the peace treaty, the Lutherans, and also part of the Orthodox population, returned, especially to Western Ingria. The proportion of Finns in Ingria kept growing, making up 41.1% of the inhabitants in 1656, 53.2% in 1661,55.2% in 1666, and 56.9% in 1671 (Inkeri 1991). In the latter year their percentage was highest in the P~ihkin~ilinna district, accounting for 86% of the population. The number of Finns was lower in West Ingria, where Ingrians and Votians prevailed. At Narva and Iamburg Russians were numerous. In 1695 the newly arrived settlers already made up 73.8% of the whole population of Ingria.
In addition to Narva there was another town - Nyen or Nevanlinna - in Ingria where the inhabitants included representatives of a number of different peoples. Already in 1583 Swedes had tried to build a stronghold by that name, but owing to continual hostilities they were not able to start construction work until 1611.In 1632 a town sprang up by the side of the stronghold at the mouth of the River Ohta, a tributary of the Neva. This town became the trading centre of Ingria and the whole of Eastern Finland, and in 1642 it was also made the administrative centre of the Ingrian Gubernia. In 1656the town was badly damaged by a Russian attack, and the administrative centre was transferred to Narva.
Two factors contributed to the fusion of these Finnish- speaking ethnic groups in Ingria: the Lutheran faith and instruction in literary Finnish at the schools in the parishes. That made the Finnish-speaking inhabitants of Ingria identify with the Finns living in Finland and they began to refer to themselves as Ingrian Finns. Nevertheless, the political separation of Ingria from Finland as a result of the Great Northern War (1700-1721), the foundation of the city of St. Petersburg (1703) on the site of the town of Nyen, and the creation of Russian villages amidst the Finnish settlements in areas inhabited almost exclusively by Finns, brought peculiar features to the development of Ingrian Finns. In the course of a hundred years Ingria remained cut off from the greater part of Finland (except for Viborg and Ladogan Karelia). The impact of the Finnish language on Ingria increased again after the annexation of Finland to Russia in 1809.
— Ott Kurs: "Ingria: The Broken Landbridge between Estonia and Finland", GeoJournal, 33.1, pp107–113, 1994. PDF
To better visualise the territorial evolution of Muscovy into Russia, and the change of hands concerning Ingermanland, a few maps

Muscovy from 1390–1525 (Ingermanland being mainly under Novgorod control, but since Muscovy swallowed Novgorod in 1478… src)

The Philips World Atlas of World History, Europe ca. 1560, rough area of Ingermanland highlighted:
— Patrick K. O'Brien (Ed): "The Philips's Atlas of World History", Philip's: London, 22005, p146.
In turn, Sweden's expansion:
src