Medi-ya grocery in Liang Court is known as the Japanese grocery - complete but I don't know if it qualifies as cheap.
They are expensive, especially anything fresh like vegetables imported from Japan.
There is a Japanese mart cum restaurant out here at West Coast Mall (formerly Ginza Plaza) that has a pretty good stock of Japanese products. Not sure on the pricing though.