Today it gets better situation for mobile data with WIFI in Japan. Actually it is not like your countries. One of the best solution is to get SIM card for your data communication on your smart phone. It is quite common to change it to local SIM card in Asia and Europe. Surely it is good cost for value.

You can connect by WIFI within only limited areas such as station, cafe, some of sightseeing spots even in Tokyo.
For the ubiquitous use, we recommend to get it with smart way as follows.

Oder it by online before you go, Get it right timing just when you come to your hotel.

You can purchase it by online such as Amazon Japan. As you know you can get all related information in English on the site and order. One thing you need to note is shipping address. Input your hotel address and send it to there. After ordering, it would be perfect to tell your hotel counter that your parcel will come. All you have to do when you reach the hotel, just get it and change new SIM card.
Have a great time in Japan with mobile.

Here are recommended sim package at this moment.

No1. b-mobile VISITOR SIM
*Unlimited usage!
This is the cheapest one for cost for amount. Other competitive SIM cards have limited data amount however this is not. The speed restriction will be released the day after your total usage in 3 past days go below 1000MB. This is not big issue anyway. This is only around $24 USD.
You can get this at Retailers such as AEON, Yodobashi camera, Bic camera and online.
When you order it by online in advance you can get it at hotel where you are going to stay.

#Type A)
Data amount unlimited
Usable period 14days
Extend the period OK
Price 2,380 JPY

They have another SIM type for longer usable period as well.
#Type B)
Data amount 5G
Usable period 21days
Price 3,480 JPY
Top-up OK

No.2 Japan Travel SIM
Data amount 2G
Usable period 3 months
Price \3,990
Top-up OK