Modi expressed his desire to travel to Lumbini, the birthplace of Lord Buddha in his future visit.
The AICC general secretary said Modi, through his temple visits, was giving out a message to voters in Karnataka of how religious he was and that he was a Hindu.
Responding to a question at a press meet, Gokhale said PM Modi had telephoned Oli, who was sworn to the post on February 15, to congratulate him and the visits were discussed then.
Following the meeting, Modi reportedly said that "all misunderstandings with Nepal are over".
In a statement Modi said the visit was part of his government's "neighbourhood first" policy and "reflects the high priority, India, and personally I, attach to our age-old, close and friendly ties with Nepal".
The land-locked country depends on India for trade and supplies.
A team of Indian experts will begin the survey for the feasibility of the rail linkages and India is also sending a team of experts to conduct feasibility survey for inland water navigation. New vigour has been added to India-Nepal relations through this visit.
"Nepal is a country of diversity and Kathmandu is a world in its own", Modi said.
During his two day visit, Modi visited three temples - Janki Temple, Muktinath Temple, and Pashupatinath Temple to strike the spiritual chord between India and Nepal.
Apart from offering prayers at temples in Nepal, the Indian leader inaugurated the Janakpur-Ayodhya bus service, a 900 MW hydropower project and announced Rs 100-crore aid to develop Janakpur city. The Investment Board Nepal (IBN) on Thursday had granted an electricity generation licence to SJVN Arun III Power Development Company to generate power from the Arun III project located in eastern Nepal.
"An auspicious start to Day 2 in Nepal! The Ramayan Circuit has vast potential".
After praying, he also greeted and interacted with people waiting at the temple premises. This will be followed by meeting with leaders of political parties of Nepal.
Earlier, Kathmandu Mayor Bidya Sundar Shakya presented symbolic key of the city to Modi, which symbolises that the door to Kathmandu would always remain open to him.