Fetch Petrol Prices Using Python and Store to Excel

An example to Fetch the daily petrol price using Python, pandas, BeautifulSoup

Imports

import requests
import pandas as pd
from bs4 import BeautifulSoup

Extract data from the Website

def getdata(url):
   r = requests.get(url)
   return r.text


htmldata = getdata("https://www.goodreturns.in/petrol-price.html")
soup = BeautifulSoup(htmldata, 'html.parser')

Declaring required Variables

# Declare string var
# Declare list
mydatastr = ''
result = []

Searching for the variable in HTML data and storing

for table in soup.find_all('tr'):
   mydatastr += table.get_text()

mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
for item in itemlist[:-5]:
   result.append(item.split("\n"))

Calling DataFrame constructor on the list using pandas

df = pd.DataFrame(result[:-8])

Saving to Excel using pandas and printing rates in console

df.to_excel("Petrol.xlsx")
print(df)

Console Output

Resulting in Excel with Fuel prices for further process

Complete code Can be found at

krishachetan89/PetrolPrices