The typhoon first made landfall in the southern part of Tokushima Prefecture at 12 pm (local time), before striking for the second time near Kobe city at around 2 pm (local time), The Japan Times reported.
The Tokyo metropolitan area may see strong winds, though the typhoon is unlikely to pass close to the capital.
Jebi has a similar trajectory to Typhoon Cimaron, which made landfall on August 23, disrupting transport links but causing limited damage and few injuries.
It is likely to make landfall on Shikoku, Japan's smallest main island, or the Kii Peninsula by midday, carrying winds of up to 16kp/h (100mph), according to the Japan Meteorological Agency.
The Japan Meteorological Agency said a wide area of Japan should be on high alert for strong winds, high waves and heavy downpours.
Japanese authorities have issued evacuation advisories for one million people and hundreds of flights have been grounded as the barrelling Typhoon Jebi pummels through the country.
At least 500 flights in western and central Japan are expected to be canceled.
Almost 600 flights were cancelled, including several global flights departing and arriving at Nagoya and Osaka, along with ferries connecting ports in western Japan.
West Japan Railway Co. halted all local services in the area's three main cities, with some subway lines in Osaka also stopped. Prime Minister Shinzo Abe canceled a scheduled trip to Kyushu, Japan's southernmost main island, to oversee the government's response to the typhoon, Kyodo said.
"I urge the Japanese people to take action to protect your lives, including preparing and evacuating early", he said.
More than 200 people were killed when Typhoon Prapiroon churned through the country less than two months ago.
The Shinkansen bullet train was operating on a reduced schedule and Universal Studios Japan, a popular amusement park near Osaka, was closed.
It has already left tens of thousands without power and authorities have urged people to move to safety.
More rain might be welcome in other parts of Japan, which has also been sweating through a deadly heat wave.