This Python script dynamically scrapes the One Piece Fandom Wiki to build a weighted network (graph) of character co-occurrences based on the chapters they appear in together.
For every chapter in which two characters appear, an edge is created between them, and the weight of that edge increases for each subsequent shared chapter appearance.
src/: Contains the main Python scraper script (scrape_one_piece.py).data/: Contains the generated network outputs. Note: The pre-generated dataset currently uploaded in this repository contains data scraped up to Chapter 1176.one_piece_cooccurrence.gexf: A complete graph file ready to be imported into Gephi for visualization.nodes.csv: Standard Id and Label metadata for the characters.edges.csv: Source, Target, and Weight connections.
Python 3.7 or higher is required.
Clone this repository and install the dependencies:
# Clone the repository
git clone https://github.com/hossam1522/one-piece-network.git
cd one-piece-network
# Create a virtual environment (optional but recommended)
python3 -m venv venv
source venv/bin/activate # On Windows use `venv\Scripts\activate`
# Install dependencies
pip install -r requirements.txtYou can run the script in two ways:
-
Dynamically fetch all chapters (Default) The script will use the MediaWiki API to find the absolute latest published chapter and scrape the entire series from Chapter 1.
python3 src/scrape_one_piece.py
-
Specify a maximum limit (Useful for testing or specific arcs) Provide a positive integer as an argument. The scraper will stop once it reaches that chapter.
python3 src/scrape_one_piece.py 50
(This will scrape from Chapter 1 to Chapter 50)
The data is sourced directly from the One Piece Fandom Wiki. To respect the server limits and bypass Cloudflare scraping protections, the script utilizes the Fandom MediaWiki parse and allpages API endpoints. It features a hardcoded 1-second delay between requests.
Note: Scraping all 1100+ chapters will take approximately 20-30 minutes to complete. Please run this responsibly.