In Japanese houses, it is the custom to take off one's shoes in the entranceway and to not wear shoes inside. Instead, house slippers or room shoes are often worn. In hotels as well, shoes are worn in the lobby or other communal spaces, but in the guest rooms, slippers are provided to wear.
Even at tourist destinations such as shrines and temples, people take off their shoes before entering the building. At such locations, shoe shelves are available, or a bag is provided for carrying shoes, so manage the shoes you have taken off in a way that is appropriate for the situation. Also, be careful not to pick up the wrong pair of shoes when leaving.
At Japanese restaurants or izakaya grill pubs, seating may require that you take off your shoes. In such places, when you have to go to the toilet, etc., shared slippers are provided so that you do not have to fetch your shoes.
