TL,DR: it's in the right ballpark (but depends on how much you weigh, what you're carrying, how fast you're going, etc.).
For the elevation alone (not counting the extra energy spent moving your legs), the energy cost to climb a height h is m * g * h where m is your mass and g ≈ 9.81 m/s2 is the gravitational acceleration. For a mass of 70 kg (≈ 11 stone, close to the average human weight plus a little extra for what you're carrying), the energy cost is thus 686 J/m = 0.16 kcal/m = 0.05 kcal/ft. 4 kcal would bring you up 23.4 m (80 ft). Even without knowing the exact step size, this is a lot less than the values painted on that staircase. However, keep in mind that this is purely the cost of elevation, and there's an extra cost in moving your legs. The physics is solid for this being a minimum, but you need to experiment with the biology to see how much energy is consumed for purposes other than strictly fighting gravity.
An actual study found that the cost of climbing stairs for a 70 kg person was about 0.15 kcal per step, for 20.3 cm (8 in) steps (going at 1.17 step/s), i.e. about 0.74 kcal/m. For comparison, going downstairs costs about one third of that. The steps in that Japanese station are probably somewhat less than 20 cm tall, so 0.1 kcal per step is in the right ballpark.
For comparison, a human standing at rest consumes about 125 W, i.e. about 0.03 kcal/s. So it seems that walking downstairs roughly doubles your energy consumption (and walking straight costs very slightly more), and walking upstairs multiplies it by about 5. That second source gives a fairly similar figure (0.16 kcal/s) for climbing stairs (but at a faster pace of almost 2 steps/s), by the way.