Skip to content

Commit 121ecdf

Browse files
dimitri-yatsenkodimitri-yatsenko
committed
added all operators and computation section.
1 parent 3260b96 commit 121ecdf

File tree

9 files changed

+789
-6
lines changed

9 files changed

+789
-6
lines changed

‎book/40-operations/040-transactions.ipynb‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1640,10 +1640,8 @@
16401640
]
16411641
},
16421642
{
1643-
"cell_type": "code",
1644-
"execution_count": null,
1643+
"cell_type": "markdown",
16451644
"metadata": {},
1646-
"outputs": [],
16471645
"source": [
16481646
"\n",
16491647
"```sql\n",

‎book/50-queries/010-QueryAlgebra.ipynb‎ renamed to ‎book/50-queries/010-operators.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
"cell_type": "markdown",
55
"metadata": {},
66
"source": [
7-
"# Relational Algebra\n",
7+
"# Query Operators\n",
88
"\n",
99
"## Defining Queries\n",
1010
"\n",

‎book/50-queries/020-restriction.ipynb‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Operator: Restriction\n",
8+
"\n",
9+
"(This is an AI-generated placeholder)\n",
10+
"\n",
11+
"The restriction operator is a core feature of DataJoint, providing a powerful and expressive way to filter and query data from tables. By allowing users to specify conditions or constraints, the restriction operator simplifies retrieving subsets of data that match specific criteria.\n",
12+
"\n",
13+
"## Overview of the Restriction Operator\n",
14+
"\n",
15+
"The restriction operator, represented by the ampersand symbol (`&`), is used to filter entries in a table based on one or more conditions. It can be applied directly to DataJoint tables or queries and works seamlessly with other DataJoint operations.\n",
16+
"\n",
17+
"### Syntax\n",
18+
"\n",
19+
"```python\n",
20+
"<Table> & <condition>\n",
21+
"```\n",
22+
"\n",
23+
"### Components\n",
24+
"1. **`<Table>`**: The DataJoint table or query to filter.\n",
25+
"2. **`<condition>`**: The filtering condition, which can be:\n",
26+
" - A dictionary of attribute-value pairs.\n",
27+
" - A string containing an SQL-like condition.\n",
28+
" - Another DataJoint query or table.\n",
29+
"\n",
30+
"## Filtering with Dictionaries\n",
31+
"\n",
32+
"The most common way to use the restriction operator is with a dictionary of attribute-value pairs. This approach matches rows where the specified attributes meet the given values.\n",
33+
"\n",
34+
"### Example\n",
35+
"\n",
36+
"```python\n",
37+
"import datajoint as dj\n",
38+
"\n",
39+
"schema = dj.Schema('example_schema')\n",
40+
"\n",
41+
"@schema\n",
42+
"class Animal(dj.Manual):\n",
43+
" definition = \"\"\"\n",
44+
" animal_id: int # Unique identifier for the animal\n",
45+
" ---\n",
46+
" species: varchar(64) # Species of the animal\n",
47+
" age: int # Age of the animal in years\n",
48+
" \"\"\"\n",
49+
"\n",
50+
"# Insert example data\n",
51+
"Animal.insert([\n",
52+
" {'animal_id': 1, 'species': 'Dog', 'age': 5},\n",
53+
" {'animal_id': 2, 'species': 'Cat', 'age': 3},\n",
54+
" {'animal_id': 3, 'species': 'Rabbit', 'age': 2}\n",
55+
"])\n",
56+
"\n",
57+
"# Restrict rows where species is 'Cat'\n",
58+
"cats = Animal & {'species': 'Cat'}\n",
59+
"print(cats.fetch())\n",
60+
"```\n",
61+
"\n",
62+
"## Filtering with SQL-like Conditions\n",
63+
"\n",
64+
"You can also specify filtering conditions as strings using SQL-like syntax. This approach provides greater flexibility for more complex queries.\n",
65+
"\n",
66+
"### Example\n",
67+
"\n",
68+
"```python\n",
69+
"# Restrict rows where age is greater than 3\n",
70+
"older_animals = Animal & 'age > 3'\n",
71+
"print(older_animals.fetch())\n",
72+
"```\n",
73+
"\n",
74+
"## Restricting with Queries\n",
75+
"\n",
76+
"The restriction operator can also be used with other DataJoint queries or tables. This allows filtering based on results from related tables or upstream queries.\n",
77+
"\n",
78+
"### Example\n",
79+
"\n",
80+
"```python\n",
81+
"@schema\n",
82+
"class Experiment(dj.Manual):\n",
83+
" definition = \"\"\"\n",
84+
" experiment_id: int # Unique experiment identifier\n",
85+
" ---\n",
86+
" animal_id: int # ID of the animal involved\n",
87+
" \"\"\"\n",
88+
"\n",
89+
"# Insert example data\n",
90+
"Experiment.insert([\n",
91+
" {'experiment_id': 1, 'animal_id': 1},\n",
92+
" {'experiment_id': 2, 'animal_id': 2}\n",
93+
"])\n",
94+
"\n",
95+
"# Restrict animals involved in experiments\n",
96+
"experiment_animals = Animal & Experiment\n",
97+
"print(experiment_animals.fetch())\n",
98+
"```\n",
99+
"\n",
100+
"## Combining Restrictions\n",
101+
"\n",
102+
"You can combine multiple restrictions using the `&` operator. This is useful for applying multiple filters to refine your query.\n",
103+
"\n",
104+
"### Example\n",
105+
"\n",
106+
"```python\n",
107+
"# Restrict rows where species is 'Dog' and age is greater than 3\n",
108+
"old_dogs = Animal & {'species': 'Dog'} & 'age > 3'\n",
109+
"print(old_dogs.fetch())\n",
110+
"```\n",
111+
"\n",
112+
"## Negating Restrictions\n",
113+
"\n",
114+
"To exclude rows matching certain conditions, you can use the negation operator (`-`).\n",
115+
"\n",
116+
"### Example\n",
117+
"\n",
118+
"```python\n",
119+
"# Exclude rows where species is 'Cat'\n",
120+
"non_cats = Animal - {'species': 'Cat'}\n",
121+
"print(non_cats.fetch())\n",
122+
"```\n",
123+
"\n",
124+
"## Best Practices\n",
125+
"\n",
126+
"1. **Use Dictionaries for Simplicity**:\n",
127+
" - When possible, use dictionaries for straightforward and readable queries.\n",
128+
"2. **Leverage SQL-like Syntax for Complex Conditions**:\n",
129+
" - Use strings for more advanced filtering needs, such as range conditions or logical operators.\n",
130+
"3. **Combine Restrictions Thoughtfully**:\n",
131+
" - Combine multiple restrictions to precisely target your data.\n",
132+
"4. **Test Incrementally**:\n",
133+
" - Test each restriction step-by-step to ensure accuracy.\n",
134+
"5. **Avoid Hardcoding**:\n",
135+
" - Use variables and parameters to dynamically construct restrictions.\n",
136+
"\n",
137+
"## Summary\n",
138+
"\n",
139+
"The restriction operator is a versatile and essential tool in DataJoint. Whether you're filtering based on simple conditions, combining multiple restrictions, or leveraging related queries, the restriction operator enables precise and efficient data retrieval. Mastering its use will greatly enhance your ability to work with DataJoint pipelines.\n",
140+
"\n"
141+
]
142+
}
143+
],
144+
"metadata": {
145+
"language_info": {
146+
"name": "python"
147+
}
148+
},
149+
"nbformat": 4,
150+
"nbformat_minor": 2
151+
}

‎book/50-queries/030-projection.ipynb‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Operator: Projection\n",
8+
"\n",
9+
"The projection operator in DataJoint is a fundamental tool for refining and transforming queries. By allowing users to select specific attributes or create derived attributes, the projection operator simplifies data retrieval and enhances the flexibility of query results.\n",
10+
"\n",
11+
"## Overview of the Projection Operator\n",
12+
"\n",
13+
"The projection operator, represented by the `.proj()` method, is used to create a new query that includes only the specified attributes from a table or an existing query. It can also be used to define new attributes based on transformations of existing ones.\n",
14+
"\n",
15+
"### Syntax\n",
16+
"\n",
17+
"```python\n",
18+
"<Table>.proj(*attributes, **renamed_attributes)\n",
19+
"```\n",
20+
"\n",
21+
"### Components\n",
22+
"1. **`*attributes`**:\n",
23+
" - A list of attributes to include in the projection.\n",
24+
" - Omitting this will include all attributes by default.\n",
25+
"2. **`**renamed_attributes`**:\n",
26+
" - Key-value pairs for renaming or creating new attributes.\n",
27+
" - The key is the new attribute name, and the value is the expression or existing attribute to use.\n",
28+
"\n",
29+
"## Selecting Specific Attributes\n",
30+
"\n",
31+
"The most common use of the projection operator is to narrow down the attributes returned by a query. This reduces the amount of data retrieved and simplifies the output.\n",
32+
"\n",
33+
"### Example\n",
34+
"\n",
35+
"```python\n",
36+
"import datajoint as dj\n",
37+
"\n",
38+
"schema = dj.Schema('example_schema')\n",
39+
"\n",
40+
"@schema\n",
41+
"class Animal(dj.Manual):\n",
42+
" definition = \"\"\"\n",
43+
" animal_id: int # Unique identifier for the animal\n",
44+
" ---\n",
45+
" species: varchar(64) # Species of the animal\n",
46+
" age: int # Age of the animal in years\n",
47+
" \"\"\"\n",
48+
"\n",
49+
"# Insert example data\n",
50+
"Animal.insert([\n",
51+
" {'animal_id': 1, 'species': 'Dog', 'age': 5},\n",
52+
" {'animal_id': 2, 'species': 'Cat', 'age': 3},\n",
53+
"])\n",
54+
"\n",
55+
"# Select only the species attribute\n",
56+
"species_only = Animal.proj('species')\n",
57+
"print(species_only.fetch())\n",
58+
"```\n",
59+
"\n",
60+
"## Renaming Attributes\n",
61+
"\n",
62+
"Projection can also rename attributes, allowing users to customize the output schema.\n",
63+
"\n",
64+
"### Example\n",
65+
"\n",
66+
"```python\n",
67+
"# Rename the 'species' attribute to 'animal_species'\n",
68+
"renamed_query = Animal.proj(animal_species='species')\n",
69+
"print(renamed_query.fetch())\n",
70+
"```\n",
71+
"\n",
72+
"## Creating Derived Attributes\n",
73+
"\n",
74+
"The projection operator enables the creation of new attributes by applying transformations or calculations to existing ones.\n",
75+
"\n",
76+
"### Example\n",
77+
"\n",
78+
"```python\n",
79+
"# Create a new attribute 'age_in_months' derived from 'age'\n",
80+
"projected_query = Animal.proj(age_in_months='age*12')\n",
81+
"print(projected_query.fetch())\n",
82+
"```\n",
83+
"\n",
84+
"## Combining Projections with Other Operators\n",
85+
"\n",
86+
"Projection is often used in combination with other DataJoint operators like restriction or joins to refine queries further.\n",
87+
"\n",
88+
"### Example\n",
89+
"\n",
90+
"```python\n",
91+
"# Restrict and project\n",
92+
"restricted_projected_query = (Animal & {'species': 'Dog'}).proj('animal_id', 'age')\n",
93+
"print(restricted_projected_query.fetch())\n",
94+
"```\n",
95+
"\n",
96+
"## Best Practices\n",
97+
"\n",
98+
"1. **Minimize Data Retrieval**:\n",
99+
" - Use projection to reduce the number of attributes retrieved, especially when working with large datasets.\n",
100+
"2. **Rename Attributes for Clarity**:\n",
101+
" - Rename attributes to make the results more readable or consistent with downstream analyses.\n",
102+
"3. **Use Derived Attributes**:\n",
103+
" - Simplify computations by creating derived attributes directly in queries.\n",
104+
"4. **Combine Projections**:\n",
105+
" - Combine projections with other operators like restriction or joins for more powerful queries.\n",
106+
"\n",
107+
"## Summary\n",
108+
"\n",
109+
"The projection operator in DataJoint is a versatile tool for customizing queries. By selecting, renaming, or deriving attributes, it enables efficient and targeted data retrieval. Mastering the projection operator will enhance your ability to work effectively with complex DataJoint pipelines.\n",
110+
"\n"
111+
]
112+
}
113+
],
114+
"metadata": {
115+
"language_info": {
116+
"name": "python"
117+
}
118+
},
119+
"nbformat": 4,
120+
"nbformat_minor": 2
121+
}

0 commit comments

Comments
 (0)