There are very strict (jail time) no-fly zones for drones all around Tokyo. And the police will likely try to stop you regardless.

This is a country that banned selfie sticks.

You may want to look at the aviation regulators in Japan. There is likely to be advice there.

I have it somewhere in my brain that getting a permit in Japan to film in public is incredibly hard.

But might also be my brain fucking with me. It does that sometimes.