Skip to content

Ncbi metadata download - #18

Open
evezeyl wants to merge 4 commits into
devfrom
NCBImeta
Open

Ncbi metadata download#18
evezeyl wants to merge 4 commits into
devfrom
NCBImeta

Conversation

@evezeyl

@evezeyl evezeyl commented Nov 11, 2022

Copy link
Copy Markdown
Contributor

Please have a fast prettifying/spelling correction and pull this to the github

Comment thread docs/source/tutorials/GetMetadataNCBI.md
Comment thread docs/source/tutorials/GetMetadataNCBI.md Outdated
Comment thread docs/source/tutorials/GetMetadataNCBI.md Outdated
Comment thread docs/source/tutorials/GetMetadataNCBI.md Outdated
Comment thread docs/source/tutorials/GetMetadataNCBI.md Outdated
Comment thread docs/source/tutorials/GetMetadataNCBI.md Outdated
@karinlag

Copy link
Copy Markdown
Member

I think I sort of understand what you are doing, but let's chat when you are back

@evezeyl

evezeyl commented Dec 2, 2022

Copy link
Copy Markdown
Contributor Author

@karinlag Can you have a second look ?
I did the improvements

@karinlag

karinlag commented Dec 5, 2022

Copy link
Copy Markdown
Member

It is almost there. You lose me with the "this"es where you introduce the yaml files :) sort that out, and I'll give it a final readthrough :)

conda activate ncbimeta
```

To use ncbimeta, you will need to use an API (application programming interface) key from NCBI.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An API is a way of asking a different computer to send you information. The key is a way for NCBI to associate you with the specific request.
Just concept sorting/clarification.

per second than you would be able to perform without it.

To get an API key from NCBI :
- login (or create an account)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log in to what?

- login (or create an account)
- account > account settings : there you can create an API key you will use - copy it and keep it safe

We will need to create a configuration file (.yalm) for each metadata download

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yaml

A trick is to create a simple search in NCBI ex:
![screenshot1](./searchfield1.png)

This search field will help you build your "metadata download configuration file" (.yalm).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yaml


Compare it to the [.yalm configuration file](./providencia_metadata.yaml)

The `.yalm` file allow to define the destination of the download (OUTPUT_DIR),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yaml?

your NCBI identification (EMAIL and API_KEY), the name of the sqlite database it will
download the metadata to (DATABASE), the tables it will create within the database
(TABLES) and the columns/fields it will create and download data to for each
table in the sqlite database (TABLE_COLUMNS, indentation: table name you want,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

identation?


So, you see that you can define your own fields, that were not provided in the other examples found in the [Biosample example](https://github.com/ktmeaton/NCBImeta/tree/master/schema).

To define those fields you need to access at which hierarchy of the `.xml` file you downloaded as helper.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence does not make sense to me.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And on reread I am completely lost here. I have no idea of how to get to the yaml file I need for the search.

conda activate NCBImeta
NCBImeta --config providencia_metadata.yaml --flat
```

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it also give csv?


To use ncbimeta, you will need to use an API (application programming interface) key from NCBI.
An API key is an authentication code that will serves to tell NCBI that you are the user
making requests to download metadata via ncbimeta software. Using an NCBI API key

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..that will serve (singular)

run the software, but it is a configuration file that specifies the search/query
we are asking the software to do: to retrieve the available metadata we are interested in,
and then the specification of the specific of fields we want to download in a database/table.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest you somehwere here show a screenshot of what this file should look like, not only link to it.

@novica
novica changed the base branch from main to dev July 2, 2024 09:33
@novica

novica commented Sep 19, 2024

Copy link
Copy Markdown
Member

Are people still working on this?

@evezeyl

evezeyl commented Sep 19, 2024 via email

Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants