Access to data based on user input

[noy2121]

Hey David,
First and foremost, thank you for this amazing project! thank you for sharing!
I'm using it for my own project (giving you all due credit) and I wonder if there's a possibility to scrape specific data using the Python code.
For example, listing specific years, competitions, and clubs as an input, combined with what stats to scrape, and get the desired csv files as an output.

Furthermore, if this capability isn't possible (yet) is there a way to fetch Champions League data?

Thank you again for your effort!

[dcaribou]

Hi @noy2121,

For example, listing specific years, competitions, and clubs as an input, combined with what stats to scrape, and get the desired csv files as an output.

Not sure I understand what is the question here. The scraper that is being used in this project it already sort of works in that way.

Furthermore, if this capability isn't possible (yet) is there a way to fetch Champions League data?

Champions League data is already being scraped and is available in the dataset.

[noy2121]

I'll double check myself and try again, for some reason I did not see any Champions League data.

Regarding my question, I will explain myself using an example. Let's say I want Players data with columns [c1,..,cn] (which can be differ from the default). Is there an option to give the code/conf file those columns and it will bring the desired data?
e.g I don't want [url, agent name, height] but I do want [2020/21_league_goals,....2023/2024_league_goals].
Its sort of replace postprocessing the csv files.

Thank You!

[dcaribou]

Hi @noy2121, sorry for the delayed response.

This project is in itself a set of configs and code to transform some raw data into curated tables. In particular the dbt project that lives in the dbt/ folder is used for this. You are welcome to modify it at your convenience to create additional columns in the existing tables.

This is valid for creating derived columns from raw data that is already available. If you want to add new data, by extending the scraper for example, you would need to get familiar with the transfermarkt-scraper first.

If you would like to contribute some of this work, I'd advice that you create a discussion (or follow-up in this one) with the details of your idea, so that we can align on it before hand 😄.

[noy2121]

Hi @dcaribou,
Thank you for getting back to me.
I initially struggled to grasp how to work with the repository, but after some persistence, I managed to figure it out and successfully utilize it as intended. Thank you for providing this resource - I really appreciate the opportunity to learn and work with it.

Noy