osint/arquivamento.qmd at main · belisards/osint · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
---
title: "Arquivamento"
---

## Recursos gerais

[Bellingcat caixa de ferramenta](https://docs.google.com/spreadsheets/d/18rtqh8EG2q1xBo2cLNyhIDuK9jrPGwYr9DI2UncoqJQ/edit#gid=1720404390)

[Witness Archive Guide](https://archiving.witness.org/archive-guide/)

[Awesome Web Archiving](https://github.com/iipc/awesome-web-archiving)

[Web Archiving software comparison](https://docs.google.com/spreadsheets/d/1FqxwaZnIhhQ7jDCC-W64NMRf5rDeh2Shx3u01MsBmTQ/edit#gid=0)

## Web

### [Archive.is](https://archive.is)
- [Extensão](https://addons.mozilla.org/en-GB/firefox/addon/wayback-machine_new/)
- [Pacote em Python](https://github.com/akamhy/waybackpy)
- [Automação com YouTube](https://github.com/bibanon/tubeup)

[Web Archive](https://web.archive.org/)

## Extensões
Use NO MÍNIMO um navegador separado, ou seja, diferente daquele que você navega.

O Chrome costuma ter boas extensões, é possível usar variantes como o [Brave](https://brave.com/) ou o [Chromium](https://www.chromium.org/chromium-projects/).

- [Conifer](https://conifer.rhizome.org/)

- [Video download helper](https://www.downloadhelper.net/)

## CLI

### wget
[https://www.gnu.org/software/wget/](https://www.gnu.org/software/wget/)

Explore os parâmetros. Exemplo:

`wget -i links.txt`

### youtube-dl
[https://github.com/ytdl-org/youtube-dl](https://github.com/ytdl-org/youtube-dl)

Não se engane com o nome, funciona também outras redes

### snscrape
[https://github.com/JustAnotherArchivist/snscrape](https://github.com/JustAnotherArchivist/snscrape)

### you-get
[https://you-get.org/](https://you-get.org/)

### twint
[https://github.com/kevctae/twint](https://github.com/kevctae/twint)

## Ferramentas

### Archive Box
[https://github.com/ArchiveBox/ArchiveBox](https://github.com/ArchiveBox/ArchiveBox)

### Auto Archiver
[https://github.com/bellingcat/auto-archiver](https://github.com/bellingcat/auto-archiver)

### Archivematica
[https://www.archivematica.org/en/](https://www.archivematica.org/en/)

### Perma.cc
[https://perma.cc](https://perma.cc)

### Sugarcube
[https://sugarcubetools.net/](https://sugarcubetools.net/)

### Google Sheets + Python

```python
!pip install snscrape
!pip install you-get

from google.colab import auth
from google.auth import default
from google.colab import drive

import gspread
import pandas as pd
import snscrape.modules.twitter as sntwitter
import os

auth.authenticate_user()
creds, _ = default()
gc = gspread.authorize(creds)

worksheet = gc.open('NOME_DA_SUA_PLANILHA').sheet1
drive.mount('/content/drive/')

df = pd.DataFrame(worksheet.get_all_values()[1:],columns=worksheet.get_all_values()[0])
```