Dev testing by bidhan-nagarro · Pull Request #217 · unicef/giga-data-ingestion

bidhan-nagarro · 2026-03-26T09:43:18Z

What type of PR is this?

build: Commits that affect build components like build tool, dependencies, project
version
chore: Miscellaneous commits (e.g. modifying .gitignore)
ci: Commits are special build commits that affect the CI/CD pipeline
docs: Commits that affect documentation only
feat: Commits that add a new feature
fix: Commits that fix a bug
perf: Commits are special refactor commits that improve performance
refactor: Commits that rewrite/restructure your code, however does not change any
behaviour
revert: Commits that revert another commit/PR, usually can be autogenerated on
GitHub or using git revert
style: Commits are special refactor commits that edit the code to comply with a
code style, linter, or formatter
test: Commits that add missing tests or correcting existing tests

Summary

What does this PR do

How to test

Instructions on how to test
Specify which files to review
etc.

Link to Jira/Asana/Airtable task (if applicable)

placeholder

Wireframe screenshot/screencap (if applicable)

placeholder

Implementation screenshot/screencap (if applicable)

placeholder

* ✨ feat: Added migration for old data without metadata json file. * 🔧 chore: Update to refer to metadata.json from the blob * 🔥 chore: Update sh file to run migrate code for older countries * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: added command for migrating old metadata * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Changed the scripts path and added to infra init jobs yaml * chore: Fixed metadata path issue. * chore: Removed code for error fix in ApprovelRequest Model * chore: Removed code for error fix in FileUpload Model * chore: Removed code for error fix in ApprovalRequest Model * refactor: Change in the init jobs to add migrate as a separate job * chore: disable migrate blob job * chore: re-enable migrate blob job * refactor: migrate script modified * refactor: migrate script modified * refactor: migrate script modified * refactor: Migrate Script fixed * refactor: Migrate Script fixed * chore: make pre-commit fixes --------- Co-authored-by: nagarro <bidhan.mondal@nagarro.com>

…ew flow - remove API call from metadata continue for school-data - add review call to POST /upload/review with same payload as submit - keep submit using existing upload API - preserve polling and existing success/navigation behavior

* ✨ feat: Added migration for old data without metadata json file. * 🔧 chore: Update to refer to metadata.json from the blob * 🔥 chore: Update sh file to run migrate code for older countries * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: added command for migrating old metadata * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Changed the scripts path and added to infra init jobs yaml * chore: Fixed metadata path issue. * chore: Removed code for error fix in ApprovelRequest Model * chore: Removed code for error fix in FileUpload Model * chore: Removed code for error fix in ApprovalRequest Model * refactor: Change in the init jobs to add migrate as a separate job * chore: disable migrate blob job * chore: re-enable migrate blob job * refactor: migrate script modified * refactor: migrate script modified * refactor: migrate script modified * refactor: Migrate Script fixed * refactor: Migrate Script fixed * chore: make pre-commit fixes --------- Co-authored-by: nagarro <bidhan.mondal@nagarro.com>

* feat: reorder columns mapping Grouped different columns into sections for easier mapping and UX * feat: add descriptions for column mapping Related to TECH-8531, adding more precise descriptions for column mapping, indicating some examples and their type. * fix: css fix and db info for tooltip have hardcoded data as fallback for first getting the information from the database * feat: pin id and name to be first items order as figma

) * feat: adding helper text as placeholder and metadata form redesign Explanations are hidden in helper text instead of being embedded in placeholders or labels Redesign form structure * fix: adjunt date input and add refiner Change date input into text input Add refiner to prevent setting past dates

* feat: removing and replacing uploaded file Adding feature of removing and replacing an uploaded file Also adjusting UI according figma design in the ticket. The template file text has not been added yet. * feat: add font and text to file removal Adding text and font to adapt to figma design

Following figma design, changed and updated submission process buttons to be more intuituve for users

* feat: create tabs and tables based on data source Created tabs in Upload page based on data source * feat: add coverage and scheemaless tabs Add coverage and scheemaless dataset tabs and adding souce to dataset * feat: filter for datasets Dataset prefiltering for Coverage and scheemaless * feat: create separate file for upload metadata * ✨ feat: Added migration for old data without metadata json file. * 🔧 chore: Update to refer to metadata.json from the blob * 🔥 chore: Update sh file to run migrate code for older countries * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: Migration file updated * 🔥 chore: added command for migrating old metadata * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Addressed Review Comments * chore: Changed the scripts path and added to infra init jobs yaml * chore: Fixed metadata path issue. * chore: Removed code for error fix in ApprovelRequest Model * chore: Removed code for error fix in FileUpload Model * chore: Removed code for error fix in ApprovalRequest Model * refactor: Change in the init jobs to add migrate as a separate job * chore: disable migrate blob job * chore: re-enable migrate blob job * refactor: migrate script modified * refactor: migrate script modified * refactor: migrate script modified * refactor: Migrate Script fixed * refactor: Migrate Script fixed * chore: make pre-commit fixes --------- Co-authored-by: Brian Musisi <brianmusisi@berkeley.edu> * feat: move filter source and dataset to the backend Moved the filtering for the tables to query the DB. TODO: Once source for Giga Sync uploads is not "null", and becomes "gigasync" we should query the Geolocation table to it --------- Co-authored-by: bidhan-nagarro <bidhan.mondal@nagarro.com> Co-authored-by: Brian Musisi <brianmusisi@berkeley.edu>

…214) * Revert "Feature/tech 6769 upload table tabs based on data source (#197)" This reverts commit 22071a0. * fix: revert only metadata changes Restoring UI changes to only revert metadata changes * feat: remove giga meter tab Remove giga meter tab until it's added as a working source

* fix: minor UI fixes Fix tooltip positioning to not be outside the table Fix Close and run in background button position * fix: minor UI additions Placeholder on dropdown Automatic / on MM/YYYY on metadata input

fix lint errors on import orders from ruff

Fixed ruff errors for azure pipelane checks

…ving to approval screen

bidhan-nagarro and others added 30 commits March 9, 2026 14:43

feat: Approval Request

31aafe6

refactor: Enable approval/rejection of individual Giga Sync uploads

2cc2ccb

refactor: Enable approval/rejection of individual Giga Sync uploads

869b476

refactor: Enable approval/rejection of individual Giga Sync uploads

96cb37d

refactor: DQ checks

d211ca0

refactor: DQ checks

529d525

refactor: DQ checks

af6062c

feat: Data Quality Checks

ef847c2

refactor: Enable approval/rejection of individual Giga Sync uploads

e210516

feat: Error table API

a5f358e

Merge branch 'main' into feature/error-table

670e849

feat: DQ changes

afc79c8

feat: Fuzzy Match

11a160e

feat: Fuzzy Logic

5e28daa

feat: Fuzzy Logic

312a517

feat: Fuzzy Logic

0bf6e43

feat: Fuzzy Logic

46850e9

feat: Fuzzy Match FE

f7d7c06

feat: added missing fuzzy modal

6277bdd

chore: fix pre-commit hook failures

d4d89b7

chore: fuzzy match migration issues fixed

352060a

feat: Added env Variables for NOCODB

558d2fa

chore: poetry.lock file updated

9b858aa

Merge branch 'main' into feature/dq

e8d7211

chore: poetry.lock file pydentic issues fixed

e801e0b

bidhan-nagarro and others added 30 commits March 18, 2026 13:50

feat: Approval Request Changes

66cdb3d

feat: adapt submit buttons to be more intuituve for users (#211)

d628cc1

Following figma design, changed and updated submission process buttons to be more intuituve for users

Fix/minor UI fixes (#215)

23c0613

* fix: minor UI fixes Fix tooltip positioning to not be outside the table Fix Close and run in background button position * fix: minor UI additions Placeholder on dropdown Automatic / on MM/YYYY on metadata input

feat: Error table changes

6685d2a

Merge branch 'main' into feature/approval-request-changes

bf75e5c

Merge branch 'main' into feature/dq

6e56ec3

Merge branch 'main' into feature/error-table

02ce128

Merge branch 'main' into feature/fuzzy-match

f9cc002

fix: fix lint errors on import orders from ruff

d3cc13d

fix lint errors on import orders from ruff

feat: restored deleted css

755932a

fix: fix ruff errors

79f0c02

Fixed ruff errors for azure pipelane checks

feat: DQ code fixes

93fd6c0

feat: DQ code fixes

ebc4f44

feat: Review Comments addressed

fb6abdc

feat: Bugs Fixed

826575f

feat: added intermediatory screen to list the uploaded list before mo…

54acfe7

…ving to approval screen

feat: Dq branch merged

554f93c

feat: Error table branch

e5eed38

feat: Fuzzy branch

edb3729

fix: linting issues

56bc370

fix: Review Comments addressed

2fbc87d

fix: review comments addressed

652dbd5

fix: pre-commit issue fixed

960ff4b

feat: enhance success component with review mode functionality

eb9eea1

fix: pydentic version fixed in poetrylock file

2ba292c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev testing#217

Dev testing#217
bidhan-nagarro wants to merge 60 commits intomainfrom
dev-testing

bidhan-nagarro commented Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

bidhan-nagarro commented Mar 26, 2026

What type of PR is this?

Summary

How to test

Link to Jira/Asana/Airtable task (if applicable)

Wireframe screenshot/screencap (if applicable)

Implementation screenshot/screencap (if applicable)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants