Python: Scrape News & Save to CSV

This project is designed for learners who know Python fundamentals and are learning to build real-world programs.

Project Description

This project uses the versatile Beautiful Soup webscraping Python library to fetch real-time news headlines from a news website and stores them in a CSV file for easy access.

We are scraping the current headlines from the Aljazeera website.

Here is the file generated by the program.

Prerequisites

Required Libraries: requests, BeautifulSoup, pandas
pip install requests beautifulsoup4 pandas
Required Files: You don’t need any files for this project.
IDE: Use any IDE.

Solution

Here you go:

import requests
from bs4 import BeautifulSoup
import pandas as pd

# Define a website that serves headlines in static HTML
URL = "https://www.aljazeera.com/news/"

# Fetch the webpage content
response = requests.get(URL, headers={"User-Agent": "Mozilla/5.0"})
soup = BeautifulSoup(response.text, "html.parser")

# Extract news headlines
headlines = soup.find_all("h3", class_="gc__title")

# Store headlines in a list
news_list = [headline.get_text(strip=True) for headline in headlines if headline.get_text(strip=True)]

# Save to CSV
df = pd.DataFrame({"Headline": news_list})
df.to_csv("news_headlines.csv", index=False, encoding="utf-8")

print(f"✅ {len(news_list)} headlines saved to 'news_headlines.csv'")