Skip to content

Commit db58fb4

Browse files
authored
Update README.md
1 parent f9f3ffa commit db58fb4

1 file changed

Lines changed: 64 additions & 38 deletions

File tree

README.md

Lines changed: 64 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,82 @@
11
# IBEX-Scraper-API
22

3-
A secure ASP.NET Core Web API that automatically scrapes **day-ahead prices from the Free Electricity Market (Day Ahead segment) on IBEX** and stores them in a structured Azure SQL database for use by external applications.
3+
A lightweight **ASP.NET Core Web API** that automatically scrapes **day-ahead electricity prices from IBEX (Independent Bulgarian Energy Exchange)** and stores them in a remote SQL Server database for use by external applications.
4+
5+
**Live API:** [link](http://ibex-scraper.somee.com/swagger/index.html) *(HTTP only – free hosting plan limitation)*
6+
7+
---
48

5-
Access it at [link](http://ibex-scraper.somee.com/swagger/index.html)
69
## ⚡ Purpose
710

811
This API was built to **decouple IBEX access from dependent systems** by providing a secure, reliable, and filterable data backend.
912

1013
It powers apps like [SunNext](https://github.com/JacksonJS12/SunNext) by allowing them to:
11-
- Access daily price data without scraping IBEX directly
12-
- Query historical electricity prices by date/hour
13-
- Build custom dashboards and energy trading strategies
14+
15+
* Access daily price data without scraping IBEX directly
16+
* Query historical electricity prices by date/hour
17+
* Build custom dashboards and energy trading strategies
18+
19+
---
1420

1521
## 📊 What It Does
1622

17-
- Scrapes **IBEX Day Ahead segment** prices for the next day
18-
- Extracts hourly energy prices (EUR/MWh)
19-
- Saves results into an Azure SQL `MarketPrices` table with:
20-
- `Date` (e.g. `day/month/year`)
21-
- `Hour` (1–24 using Central European Time – CET Time Zone)
22-
- `PricePerMWh` (BGN/MWh)
23+
* Scrapes **IBEX Day Ahead** prices for the next day
24+
* Extracts hourly energy prices (BGN/MWh)
25+
* Saves results into a `MarketPrices` table with:
26+
27+
* `Date` (format: `dd/MM/yyyy`)
28+
* `Hour` (1–24 using EET/EEST timezone)
29+
* `PricePerMWh` (BGN/MWh)
30+
31+
---
2332

2433
## 🕒 Scheduling
2534

26-
- Powered by an **Azure Logic App** that triggers daily at **14:00 BG time (UTC+3)**
27-
- Calls the `POST /api/Scrape/scrape-and-save` endpoint
28-
- Only stores new data (prevents duplicate entries for the same day)
29-
30-
<center><img width="394" height="356" alt="image" src="https://github.com/user-attachments/assets/897ffe35-0acf-4eba-b825-cbbb05ec982d" /></center>
35+
* Prices are updated daily after **14:00 BG time (UTC+3)** when IBEX publishes the next day’s market prices
36+
* Data scraping is triggered by calling:
37+
38+
```
39+
POST /api/Scrape/scrape-and-save
40+
```
41+
* Only stores **new** data (prevents duplicate entries for the same date/hour)
42+
43+
---
3144

3245
## 🧱 Tech Stack
3346

34-
- **Backend:** ASP.NET Core Web API (.NET 8)
35-
- **Scraper:** AngleSharp
36-
- **Database:** Azure SQL
37-
- **Scheduler:** Azure Logic Apps
38-
- **Deployment:** Azure App Service
39-
- **CI/CD:** GitHub Actions
40-
41-
📁 Project Structure
42-
IBEX-Scraper-API/ <br/>
43-
├── Controllers/ <br/>
44-
│     └── ScrapeController.cs <br/>
45-
├── Services/ <br/>
46-
│     └── IbexScraper/ <br/>
47-
│         └── IbexScraperService.cs <br/>
48-
│         └── IIbexScraperService.cs <br/>
49-
├── Data/ <br/>
50-
│     └── Migrations/ <br/>
51-
│     └── Models/ <br/>
52-
│         └── MarketPrice.cs <br/>
53-
│     └── AppDbContext.cs <br/>
54-
│     └── AppDbContextFactory.cs <br/>
55-
└── Program.cs <br/>
47+
* **Backend:** ASP.NET Core Web API (.NET 8)
48+
* **Scraper:** AngleSharp
49+
* **Database:** MS SQL (Somee.com hosting)
50+
* **Deployment:** FTP to Somee.com (Free Hosting Plan, HTTP-only)
51+
52+
---
53+
54+
## 🌐 Deployment Notes (Somee Free Plan)
55+
56+
* Hosted on **Somee.com** free plan
57+
* **HTTP only** – no HTTPS due to free hosting limitations
58+
* API runs at: [http://ibex-scraper.somee.com](http://ibex-scraper.somee.com)
59+
* Connection string stored in `appsettings.Production.json` on the server
60+
* Deployment via FTP to `/www.ibex-scraper.somee.com`
61+
62+
---
63+
64+
## 📁 Project Structure
65+
66+
```
67+
IBEX-Scraper-API/
68+
├── Controllers/
69+
│ └── ScrapeController.cs
70+
├── Services/
71+
│ └── IbexScraper/
72+
│ ├── IbexScraperService.cs
73+
│ └── IIbexScraperService.cs
74+
├── Data/
75+
│ ├── Migrations/
76+
│ ├── Models/
77+
│ │ └── MarketPrice.cs
78+
│ ├── AppDbContext.cs
79+
│ └── AppDbContextFactory.cs
80+
└── Program.cs
81+
```
5682

0 commit comments

Comments
 (0)