Skip to content

:chores: Moved dataset import/export to its own feature folder #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jpggvilaca wants to merge 9 commits into from
Closed

:chores: Moved dataset import/export to its own feature folder #249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e66bf5d
Moved dataset ie to features folder
jpggvilaca May 20, 2025
32d47fb
Fix tests
jpggvilaca May 20, 2025
294712c
Fix lint
jpggvilaca May 20, 2025
580912e
Remove duplicated file
jpggvilaca May 20, 2025
e4e120d
Move providers
jpggvilaca May 23, 2025
0907ccb
Move job utils to shared folder
jpggvilaca May 26, 2025
17bdbeb
Fix rebase conflicts
jpggvilaca Jun 4, 2025
012112b
Fix 1 more conflict
jpggvilaca Jun 4, 2025
047e6b0
Fix cyclic dependency
jpggvilaca Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions web_ui/src/core/datasets/hooks/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
// Copyright (C) 2022-2025 Intel Corporation
// LIMITED EDGE SOFTWARE DISTRIBUTION LICENSE

import { getFailedJobMessage } from '@geti/core/src/services/utils';
import { AxiosError, HttpStatusCode } from 'axios';
import isFunction from 'lodash-es/isFunction';
import { isFunction } from 'lodash-es';

import { getFailedJobMessage } from '../../../../packages/core/src/services/utils';
import { IntervalJobHandlers } from '../../../features/dataset-import/types/dataset-import.interface';
Copy link

Copilot AI May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Core utilities should not depend on feature-level types to avoid cyclic or inverted dependencies. Consider moving IntervalJobHandlers to a shared core or common package.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@dwesolow dwesolow May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. We don't want to import anything from features/pages to core.

Copy link
Contributor Author

@jpggvilaca jpggvilaca May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree! Moved to shared

import {
isJobCancel,
isJobDone,
isJobFailed,
isJobSettled,
} from '../../../shared/components/header/jobs-management/utils';
import { JobGeneralProps } from '../../jobs/jobs.interface';
import { IntervalJobHandlers } from './dataset-import.interface';

export const getIntervalJobHandlers =
<T extends JobGeneralProps>({
Expand Down
57 changes: 57 additions & 0 deletions web_ui/src/features/dataset-export/components/dataset-export.module.scss
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
.exportStatusContainer {
position: relative;
margin: var(--spectrum-global-dimension-size-250) 0;
background: var(--spectrum-global-color-gray-75);
}

.exportStatusTitle {
margin: 0;
margin-bottom: var(--spectrum-global-dimension-size-100);
font-size: var(--spectrum-global-dimension-size-200);
font-weight: normal;
}

.exportDialogBody {
padding: var(--spectrum-global-dimension-size-250) !important;
background: var(--spectrum-global-color-gray-50) !important;
}

.smallHeader {
font-size: var(--spectrum-global-dimension-font-size-75);
color: var(--spectrum-global-color-gray-700);
}

.linkButton {
&:not(:hover) {
color: var(--energy-blue);

> svg {
fill: var(--energy-blue) !important;
}
}
}

.messages {
display: flex;
align-items: center;
margin-top: var(--spectrum-global-dimension-size-300);
margin-bottom: var(--spectrum-global-dimension-size-115);

svg {
fill: var(--spectrum-semantic-notice-color-icon);
margin-right: var(--spectrum-global-dimension-size-150);
}
}

.messageList {
position: relative;
list-style-type: disc;

li {
margin-left: var(--spectrum-global-dimension-size-600);

&::marker {
color: var(--spectrum-alias-icon-color);
}
}
}
12 changes: 6 additions & 6 deletions ...xport-dataset/dataset-statistics.test.tsx → ...rt/components/dataset-statistics.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@

import { render, screen } from '@testing-library/react';

import { useDatasetStatistics } from '../../../../../core/statistics/hooks/use-dataset-statistics.hook';
import { useDatasetIdentifier } from '../../../../annotator/hooks/use-dataset-identifier.hook';
import { useMedia } from '../../../../media/providers/media-provider.component';
import { useDatasetStatistics } from '../../../core/statistics/hooks/use-dataset-statistics.hook';
import { useDatasetIdentifier } from '../../../pages/annotator/hooks/use-dataset-identifier.hook';
import { useMedia } from '../../../pages/media/providers/media-provider.component';
import { ExportDatasetStatistics } from './export-dataset-dialog.component';

jest.mock('../../../../media/providers/media-provider.component', () => ({
jest.mock('../../../pages/media/providers/media-provider.component', () => ({
useMedia: jest.fn(),
}));

jest.mock('../../../../annotator/hooks/use-dataset-identifier.hook', () => ({
jest.mock('../../../pages/annotator/hooks/use-dataset-identifier.hook', () => ({
useDatasetIdentifier: jest.fn(),
}));

jest.mock('../../../../../core/statistics/hooks/use-dataset-statistics.hook', () => ({
jest.mock('../../../core/statistics/hooks/use-dataset-statistics.hook', () => ({
useDatasetStatistics: jest.fn(),
}));

Expand Down
24 changes: 12 additions & 12 deletions ...taset/export-dataset-dialog.component.tsx → ...nents/export-dataset-dialog.component.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,19 +23,19 @@ import {
import { LinkOut } from '@geti/ui/icons';
import { OverlayTriggerState } from '@react-stately/overlays';

import { DOCS_BASE_URL } from '../../../../../core/const';
import { DOMAIN } from '../../../../../core/projects/core.interface';
import { ExportFormats } from '../../../../../core/projects/dataset.interface';
import { useDatasetStatistics } from '../../../../../core/statistics/hooks/use-dataset-statistics.hook';
import { isNotCropDomain, openNewTab, pluralize } from '../../../../../shared/utils';
import { useDatasetIdentifier } from '../../../../annotator/hooks/use-dataset-identifier.hook';
import { useMedia } from '../../../../media/providers/media-provider.component';
import { useExportDataset } from '../../../hooks/use-export-dataset.hook';
import { useProject } from '../../../providers/project-provider/project-provider.component';
import { DOCS_BASE_URL } from '../../../core/const';
import { DOMAIN } from '../../../core/projects/core.interface';
import { ExportFormats } from '../../../core/projects/dataset.interface';
import { useDatasetStatistics } from '../../../core/statistics/hooks/use-dataset-statistics.hook';
import { useDatasetIdentifier } from '../../../pages/annotator/hooks/use-dataset-identifier.hook';
import { useMedia } from '../../../pages/media/providers/media-provider.component';
import { useProject } from '../../../pages/project-details/providers/project-provider/project-provider.component';
import { isNotCropDomain, openNewTab, pluralize } from '../../../shared/utils';
import { useExportDataset } from '../hooks/use-export-dataset.hook';
import { AVAILABLE_FORMATS, ExportFormatDetails, isDatumaroFormat } from '../utils/utils';
import { ExportDatasetMessage } from './export-dataset-message.component';
import { AVAILABLE_FORMATS, ExportFormatDetails, isDatumaroFormat } from './utils';

import classes from '../project-dataset.module.scss';
import classes from './dataset-export.module.scss';

enum SaveVideoFormat {
NATIVE = 'native',
Expand Down Expand Up @@ -185,7 +185,7 @@ export const ExportDatasetDialog = ({ triggerState, datasetName }: ExportDataset
Convert frames to individual images
</Radio>
</RadioGroup>
<Divider size='S' UNSAFE_className={classes.divider} />
<Divider size='S' />
</>
)}
<RadioGroup
Expand Down
36 changes: 19 additions & 17 deletions ...rt-dataset/export-dataset-dialog.test.tsx → ...components/export-dataset-dialog.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,37 +5,37 @@ import { defaultTheme, Provider } from '@geti/ui';
import { useOverlayTriggerState } from '@react-stately/overlays';
import { fireEvent, render, RenderResult, screen } from '@testing-library/react';

import { createInMemoryMediaService } from '../../../../../core/media/services/in-memory-media-service/in-memory-media-service';
import { MediaService } from '../../../../../core/media/services/media-service.interface';
import { DOMAIN } from '../../../../../core/projects/core.interface';
import { ExportFormats } from '../../../../../core/projects/dataset.interface';
import { getMockedProject } from '../../../../../test-utils/mocked-items-factory/mocked-project';
import { RequiredProviders } from '../../../../../test-utils/required-providers-render';
import { MediaProvider } from '../../../../media/providers/media-provider.component';
import { useProject } from '../../../providers/project-provider/project-provider.component';
import { createInMemoryMediaService } from '../../../core/media/services/in-memory-media-service/in-memory-media-service';
import { MediaService } from '../../../core/media/services/media-service.interface';
import { DOMAIN } from '../../../core/projects/core.interface';
import { ExportFormats } from '../../../core/projects/dataset.interface';
import { MediaProvider } from '../../../pages/media/providers/media-provider.component';
import { useProject } from '../../../pages/project-details/providers/project-provider/project-provider.component';
import { getMockedProject } from '../../../test-utils/mocked-items-factory/mocked-project';
import { RequiredProviders } from '../../../test-utils/required-providers-render';
import { ExportDatasetDialog } from './export-dataset-dialog.component';

jest.mock('../../../providers/project-provider/project-provider.component', () => ({
...jest.requireActual('../../../providers/project-provider/project-provider.component'),
jest.mock('../../../pages/project-details/providers/project-provider/project-provider.component', () => ({
...jest.requireActual('../../../pages/project-details/providers/project-provider/project-provider.component'),
useProject: jest.fn(),
}));

const mockDatasetId = '321';
const mockWorkspaceId = '123';
jest.mock('../../../../annotator/hooks/use-dataset-identifier.hook', () => ({
...jest.requireActual('../../../../annotator/hooks/use-dataset-identifier.hook'),
jest.mock('../../../pages/annotator/hooks/use-dataset-identifier.hook', () => ({
...jest.requireActual('../../../pages/annotator/hooks/use-dataset-identifier.hook'),
useDatasetIdentifier: () => ({ workspaceId: mockWorkspaceId, datasetId: mockDatasetId }),
}));

const mockAddNotification = jest.fn();
jest.mock('../../../../../notification/notification.component', () => ({
...jest.requireActual('../../../../../notification/notification.component'),
jest.mock('../../../notification/notification.component', () => ({
...jest.requireActual('../../../notification/notification.component'),
useNotification: () => ({ addNotification: mockAddNotification }),
}));

const mockPrepareExportDataset = jest.fn();
jest.mock('../../../hooks/use-export-dataset.hook', () => ({
...jest.requireActual('../../../hooks/use-export-dataset.hook'),
jest.mock('../hooks/use-export-dataset.hook', () => ({
...jest.requireActual('../hooks/use-export-dataset.hook'),
useExportDataset: () => ({
prepareExportDatasetJob: {
mutate: mockPrepareExportDataset,
Expand All @@ -61,7 +61,9 @@ describe('ExportDatasetDialog', () => {
mediaService?: MediaService;
}): Promise<RenderResult> => {
jest.mocked(useProject).mockReturnValue({
...jest.requireActual('../../../providers/project-provider/project-provider.component'),
...jest.requireActual(
'../../../pages/project-details/providers/project-provider/project-provider.component'
),
project: getMockedProject({ domains }),
isTaskChainProject,
});
Expand Down
10 changes: 5 additions & 5 deletions ...set/export-dataset-download.component.tsx → ...nts/export-dataset-download.component.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ import { useApplicationServices } from '@geti/core/src/services/application-serv
import { Button, ButtonGroup, Flex, Heading, Text, View } from '@geti/ui';
import { isEmpty, isString } from 'lodash-es';

import { ExportDatasetLSData } from '../../../../../core/projects/dataset.interface';
import { NOTIFICATION_TYPE } from '../../../../../notification/notification-toast/notification-type.enum';
import { useNotification } from '../../../../../notification/notification.component';
import { downloadFile, getDownloadNotificationMessage, getFileSize } from '../../../../../shared/utils';
import { ExportDatasetLSData } from '../../../core/projects/dataset.interface';
import { NOTIFICATION_TYPE } from '../../../notification/notification-toast/notification-type.enum';
import { useNotification } from '../../../notification/notification.component';
import { downloadFile, getDownloadNotificationMessage, getFileSize } from '../../../shared/utils';

import classes from '../project-dataset.module.scss';
import classes from './dataset-export.module.scss';

interface ExportDatasetDownloadProps {
onCloseDownload: (id: string) => void;
Expand Down
14 changes: 7 additions & 7 deletions ...-dataset/export-dataset-download.test.tsx → ...mponents/export-dataset-download.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@
import { defaultTheme, Provider } from '@geti/ui';
import { fireEvent, RenderResult, screen } from '@testing-library/react';

import { ExportDatasetLSData, ExportFormats } from '../../../../../core/projects/dataset.interface';
import { downloadFile, getDownloadNotificationMessage } from '../../../../../shared/utils';
import { providersRender as render } from '../../../../../test-utils/required-providers-render';
import { ExportDatasetLSData, ExportFormats } from '../../../core/projects/dataset.interface';
import { downloadFile, getDownloadNotificationMessage } from '../../../shared/utils';
import { providersRender as render } from '../../../test-utils/required-providers-render';
import { ExportDatasetDownload } from './export-dataset-download.component';

jest.mock('../../../hooks/use-export-dataset.hook', () => ({
...jest.requireActual('../../../hooks/use-export-dataset.hook'),
jest.mock('../hooks/use-export-dataset.hook', () => ({
...jest.requireActual('../hooks/use-export-dataset.hook'),
useExportDataset: jest.fn(() => ({ exportDatasetStatus: {} })),
}));
jest.mock('../../../../../shared/utils', () => ({
...jest.requireActual('../../../../../shared/utils'),
jest.mock('../../../shared/utils', () => ({
...jest.requireActual('../../../shared/utils'),
downloadFile: jest.fn(),
}));

Expand Down
10 changes: 5 additions & 5 deletions ...aset/export-dataset-message.component.tsx → ...ents/export-dataset-message.component.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@
import { Text } from '@geti/ui';
import { Alert } from '@geti/ui/icons';

import { ExportFormats } from '../../../../../core/projects/dataset.interface';
import { isClassificationDomain, isRotatedDetectionDomain } from '../../../../../core/projects/domains';
import { ProjectProps } from '../../../../../core/projects/project.interface';
import { ExportFormats } from '../../../core/projects/dataset.interface';
import { isClassificationDomain, isRotatedDetectionDomain } from '../../../core/projects/domains';
import { ProjectProps } from '../../../core/projects/project.interface';
import {
CLASSIFICATION_MESSAGE,
EXPORT_VIDEO_NOT_SUPPORTED_MESSAGE,
ROTATED_BOUNDING_MESSAGE,
TASK_CHAIN_MESSAGE,
} from './utils';
} from '../utils/utils';

import classes from '../project-dataset.module.scss';
import classes from './dataset-export.module.scss';

interface ExportDatasetMessageProps {
project: ProjectProps;
Expand Down
10 changes: 5 additions & 5 deletions ...t-dataset/export-dataset-message.test.tsx → ...omponents/export-dataset-message.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@

import { render, RenderResult, screen } from '@testing-library/react';

import { DOMAIN } from '../../../../../core/projects/core.interface';
import { ExportFormats } from '../../../../../core/projects/dataset.interface';
import { ProjectProps } from '../../../../../core/projects/project.interface';
import { ExportDatasetMessage } from './export-dataset-message.component';
import { DOMAIN } from '../../../core/projects/core.interface';
import { ExportFormats } from '../../../core/projects/dataset.interface';
import { ProjectProps } from '../../../core/projects/project.interface';
import {
CLASSIFICATION_MESSAGE,
EXPORT_VIDEO_NOT_SUPPORTED_MESSAGE,
ROTATED_BOUNDING_MESSAGE,
TASK_CHAIN_MESSAGE,
} from './utils';
} from '../utils/utils';
import { ExportDatasetMessage } from './export-dataset-message.component';

describe('ExportDatasetMessage', () => {
beforeEach(() => {
Expand Down
4 changes: 2 additions & 2 deletions ...export-dataset-notification.component.tsx → ...export-dataset-notification.component.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import { useEffect } from 'react';
import { View } from '@geti/ui';
import { OverlayTriggerState, useOverlayTriggerState } from '@react-stately/overlays';

import { DatasetIdentifier, ExportDatasetLSData } from '../../../../../core/projects/dataset.interface';
import { useLocalStorageExportDataset } from '../../../hooks/use-local-storage-export-dataset.hook';
import { DatasetIdentifier, ExportDatasetLSData } from '../../../core/projects/dataset.interface';
import { useLocalStorageExportDataset } from '../hooks/use-local-storage-export-dataset.hook';
import { ExportDatasetDownload } from './export-dataset-download.component';
import { ExportDatasetStatusJob } from './export-dataset-status-job.component';

Expand Down
14 changes: 7 additions & 7 deletions ...aset/export-dataset-notification.test.tsx → ...ents/export-dataset-notification.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ import { useOverlayTriggerState } from '@react-stately/overlays';
import { RenderResult, screen, waitFor } from '@testing-library/react';
import { userEvent } from '@testing-library/user-event';

import { JobState } from '../../../../../core/jobs/jobs.const';
import { DatasetIdentifier, ExportFormats } from '../../../../../core/projects/dataset.interface';
import { createInMemoryProjectService } from '../../../../../core/projects/services/in-memory-project-service';
import { getMockedDatasetIdentifier } from '../../../../../test-utils/mocked-items-factory/mocked-identifiers';
import { getMockedDatasetExportJob } from '../../../../../test-utils/mocked-items-factory/mocked-jobs';
import { providersRender as render } from '../../../../../test-utils/required-providers-render';
import { JobState } from '../../../core/jobs/jobs.const';
import { DatasetIdentifier, ExportFormats } from '../../../core/projects/dataset.interface';
import { createInMemoryProjectService } from '../../../core/projects/services/in-memory-project-service';
import { getMockedDatasetIdentifier } from '../../../test-utils/mocked-items-factory/mocked-identifiers';
import { getMockedDatasetExportJob } from '../../../test-utils/mocked-items-factory/mocked-jobs';
import { providersRender as render } from '../../../test-utils/required-providers-render';
import { ExportDatasetNotification } from './export-dataset-notification.component';

const mockDatasetIdentifier: DatasetIdentifier = getMockedDatasetIdentifier({
Expand All @@ -30,7 +30,7 @@ const mockExportDatasetLSData = {
const mockUpdateLsExportDataset = jest.fn();
const mockRemoveDatasetLsByDatasetId = jest.fn();
const mockGetDatasetLsByDatasetId = jest.fn();
jest.mock('../../../hooks/use-local-storage-export-dataset.hook', () => ({
jest.mock('../hooks/use-local-storage-export-dataset.hook', () => ({
useLocalStorageExportDataset: () => ({
getDatasetLsByDatasetId: mockGetDatasetLsByDatasetId,
updateLsExportDataset: mockUpdateLsExportDataset,
Expand Down
18 changes: 9 additions & 9 deletions ...t/export-dataset-status-job.component.tsx → ...s/export-dataset-status-job.component.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ import { FC, ReactNode, useState } from 'react';

import { Divider, Flex, Heading, LoadingIndicator, Text, View } from '@geti/ui';

import { getJobActiveStep } from '../../../../../core/jobs/utils';
import { DatasetIdentifier, ExportDatasetLSData, ExportFormats } from '../../../../../core/projects/dataset.interface';
import { NOTIFICATION_TYPE } from '../../../../../notification/notification-toast/notification-type.enum';
import { useNotification } from '../../../../../notification/notification.component';
import { JobProgress } from '../../../../../shared/components/header/jobs-management/job-progress.component';
import { ThinProgressBar } from '../../../../../shared/components/thin-progress-bar/thin-progress-bar.component';
import { formatDownloadUrl } from '../../../../../shared/utils';
import { useExportDataset } from '../../../hooks/use-export-dataset.hook';
import { getJobActiveStep } from '../../../core/jobs/utils';
import { DatasetIdentifier, ExportDatasetLSData, ExportFormats } from '../../../core/projects/dataset.interface';
import { NOTIFICATION_TYPE } from '../../../notification/notification-toast/notification-type.enum';
import { useNotification } from '../../../notification/notification.component';
import { JobProgress } from '../../../shared/components/header/jobs-management/job-progress.component';
import { ThinProgressBar } from '../../../shared/components/thin-progress-bar/thin-progress-bar.component';
import { formatDownloadUrl } from '../../../shared/utils';
import { useExportDataset } from '../hooks/use-export-dataset.hook';

import classes from '../project-dataset.module.scss';
import classes from './dataset-export.module.scss';

interface ExportDatasetStatusJobProps {
datasetIdentifier: DatasetIdentifier;
Expand Down
Loading
Loading