Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,8 @@ APP_FILE_STORAGE_S3_ROOT_URL=https://storage.yandexcloud.net/arm-supervisor
- APP_FILE_STORAGE_S3_USE_SSL - использовать SSL для подключения
- APP_FILE_STORAGE_S3_ROOT_URL - адрес S3 хранилища, включая бакет
- APP_FILE_STORAGE_S3_REGION - регион S3 хранилища

### Параметры загрузки файла в определенное хранилище

В проекте по токену GET_FILE_STORAGE_PARAMS_USE_CASE_TOKEN можно положить в DI-контейнер юзкейс, возвращающий параметры загрузки в определенный тип хранилища для конкретного fileType.
Данный юзкейс будет вызван при загрузке файла в FileService и FileImageService
3 changes: 3 additions & 0 deletions src/domain/dtos/FileSaveDto.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ export class FileSaveDto {
@ExtendField(FileModel)
folder: string;

@ExtendField(FileModel)
fileType?: string;

@ExtendField(FileModel)
md5: string;
}
14 changes: 9 additions & 5 deletions src/domain/interfaces/IFileStorage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ import {IFileReadable} from './IFileReadable';
import {IFileWritable} from './IFileWritable';

export interface IFileStorage {
init(config: any)
read(file: IFileReadable): Promise<Buffer>
write(file: IFileWritable, source: Readable | Buffer): Promise<FileWriteResult>
getUrl(file: IFileReadable): string
deleteFile(fileName: string): void | Promise<void>;
init(config: any),
read(file: IFileReadable): Promise<Buffer>,
write(
file: IFileWritable,
source: Readable | Buffer,
fileStorageParams?: Record<string, any> | null,
): Promise<FileWriteResult>,
getUrl(file: IFileReadable): string,
deleteFile(fileName: string): void | Promise<void>,
}
8 changes: 7 additions & 1 deletion src/domain/models/FileModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import {
StringField,
CreateTimeField, IntegerField, UidField, EnumField,
} from '@steroidsjs/nest/infrastructure/decorators/fields';
import {FileImageModel} from './FileImageModel';
import FileStorageEnum from '../enums/FileStorageEnum';
import {FileImageModel} from './FileImageModel';

/**
* Файлы
Expand Down Expand Up @@ -66,6 +66,12 @@ export class FileModel {
})
folder: string;

@StringField({
label: 'Тип файла',
nullable: true,
})
fileType: string;
Comment on lines +69 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Миграции для модели File у нас создаются же в проектах? Тогда получается что нужно в MigrationGuide написать чтобы запустили генерацию миграций после накатывания этого изменения.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Напишу, только MigrationGuide во время выпуска новой версии заполняется обычно. Или предлагаешь в рамках PR сразу набрасывать наброски?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

В целом он только с выпуском новой версии нужен, но тогда надо как-то так сделать, чтобы это не забылось во время проверки новой версии...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ну то есть добавить набросок в MigrationGuide? В отдельный раздел какой-нибудь типа Pending

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Или может создать задачу, в которой прописать что нужно обновить версию и добавить конкретную инфу в MigrationGuide?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я новые версии без задач оформляю обычно, прохожу по истории коммитов и всё записываю в описание версии и MigrationGuide

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Получается, тогда в коммите это стоит отметить? Если по-правильному, то через BREAKING CHANGE: ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Оно не особо то и BREAKING 🌚
А так я планировал сразу новую версию и выкатить, мне на проекте это обновление нужно. Так что не забуду

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Оно не особо то и BREAKING 🌚

Да, но тогда надо придумать такие изменения, которые по факту не будут работать без дополнительных действий.

А так я планировал сразу новую версию и выкатить, мне на проекте это обновление нужно. Так что не забуду

Я бы отталкивался от процессов, а не от конкретных людей 🌚. Так что если не сложно, то зафиксируй это, хотя бы в задаче )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

зафиксируй это, хотя бы в задаче )

оставил в задаче комментарий


@CreateTimeField({
label: 'Создан',
})
Expand Down
37 changes: 26 additions & 11 deletions src/domain/services/FileImageService.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import * as sharp from 'sharp';
import {DataMapper} from '@steroidsjs/nest/usecases/helpers/DataMapper';
import {ContextDto} from '@steroidsjs/nest/usecases/dtos/ContextDto';
import {Inject, Optional} from '@nestjs/common';
import {IFileImageRepository} from '../interfaces/IFileImageRepository';
import {FileImageModel} from '../models/FileImageModel';
import {FileConfigService} from './FileConfigService';
import {FileModel} from '../models/FileModel';
import {FileHelper} from '../helpers/FileHelper';
import FilePreviewEnum from '../enums/FilePreviewEnum';
Expand All @@ -12,8 +12,13 @@ import {SharpHelper} from '../helpers/SharpHelper';
import {IFilePreviewOptions} from '../interfaces/IFilePreviewOptions';
import {FileRemovedEventDto} from '../dtos/events/FileRemovedEventDto';
import {IEventEmitter} from '../interfaces/IEventEmitter';
import { IFileStorageFactory } from '../interfaces/IFileStorageFactory';
import {IFileStorageFactory} from '../interfaces/IFileStorageFactory';
import FileStorageEnum from '../enums/FileStorageEnum';
import {
GET_FILE_STORAGE_PARAMS_USE_CASE_TOKEN,
IGetFileStorageParamsUseCase,
} from '../../usecases/getFileStorageParams/interfaces/IGetFileStorageParamsUseCase';
import {FileConfigService} from './FileConfigService';

const SVG_MIME_TYPE = 'image/svg+xml';

Expand All @@ -23,6 +28,9 @@ export class FileImageService {
protected readonly fileConfigService: FileConfigService,
protected readonly fileStorageFactory: IFileStorageFactory,
protected readonly eventEmitter: IEventEmitter,
@Optional()
@Inject(GET_FILE_STORAGE_PARAMS_USE_CASE_TOKEN)
protected readonly getFileStorageParamsUseCase?: IGetFileStorageParamsUseCase,
) {
}

Expand Down Expand Up @@ -113,15 +121,22 @@ export class FileImageService {
});

if (hasChanges) {
await this.fileStorageFactory.get(file.storageName).write(
DataMapper.create<FileSaveDto>(FileSaveDto, {
uid: file.uid,
folder: imageModel.folder,
fileName: imageModel.fileName,
fileMimeType: file.fileMimeType,
}),
data,
);
const fileStorageParams = this.getFileStorageParamsUseCase
? await this.getFileStorageParamsUseCase.handle(file.fileType, file.storageName)
: null;

await this.fileStorageFactory
.get(file.storageName)
.write(
DataMapper.create<FileSaveDto>(FileSaveDto, {
uid: file.uid,
folder: imageModel.folder,
fileName: imageModel.fileName,
fileMimeType: file.fileMimeType,
}),
data,
fileStorageParams,
);
}

return this.repository.create(imageModel);
Expand Down
14 changes: 12 additions & 2 deletions src/domain/services/FileService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ import {FileRemovedEventDto} from '../dtos/events/FileRemovedEventDto';
import {IFileTypeService} from '../interfaces/IFileTypeService';
import {IFileStorageFactory} from '../interfaces/IFileStorageFactory';
import FileStorageEnum from '../enums/FileStorageEnum';
import {
IGetFileStorageParamsUseCase,
} from '../../usecases/getFileStorageParams/interfaces/IGetFileStorageParamsUseCase';
import {FileConfigService} from './FileConfigService';
import {FileImageService} from './FileImageService';

Expand All @@ -38,13 +41,14 @@ function isFileExpressOrLocalSource(

export class FileService extends ReadService<FileModel> {
constructor(
public repository: IFileRepository,
protected readonly repository: IFileRepository,
protected readonly fileImageService: FileImageService,
protected readonly fileConfigService: FileConfigService,
protected readonly fileStorageFactory: IFileStorageFactory,
protected readonly eventEmitter: IEventEmitter,
protected readonly fileTypeService: IFileTypeService,
public validators: IValidator[],
protected readonly getFileStorageParamsUseCase?: IGetFileStorageParamsUseCase,
) {
super();
}
Expand Down Expand Up @@ -122,8 +126,14 @@ export class FileService extends ReadService<FileModel> {
// Get file stream from source
const stream = await this.createStreamFromSource(options.source);

const fileStorageParams = this.getFileStorageParamsUseCase
? await this.getFileStorageParamsUseCase.handle(options.fileType, options.storageName)
: null;

// Save original file via storage
const writeResult = await this.fileStorageFactory.get(options.storageName).write(fileDto, stream);
const writeResult = await this.fileStorageFactory
.get(options.storageName)
.write(fileDto, stream, fileStorageParams);

// Delete temporary file
const shouldDeleteTemporaryFile = !this.fileConfigService.saveTemporaryFileAfterUpload;
Expand Down
16 changes: 14 additions & 2 deletions src/domain/storages/FileLocalStorage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import {IFileLocalStorage} from '../interfaces/IFileLocalStorage';
import {IFileReadable} from '../interfaces/IFileReadable';
import {IFileWritable} from '../interfaces/IFileWritable';

const DEFAULT_FILE_ENCODING: BufferEncoding = 'utf8';

export class FileLocalStorage implements IFileLocalStorage {
/**
* Absolute path to root user files dir
Expand All @@ -34,7 +36,11 @@ export class FileLocalStorage implements IFileLocalStorage {
return fs.promises.readFile(filePath);
}

public async write(file: IFileWritable, source: Readable | Buffer): Promise<FileWriteResult> {
public async write(
file: IFileWritable,
source: Readable | Buffer,
fileStorageParams: Record<string, any> | null = {},
): Promise<FileWriteResult> {
const dir = join(...[this.rootPath, file.folder].filter(Boolean));

// Create dir
Expand All @@ -43,7 +49,13 @@ export class FileLocalStorage implements IFileLocalStorage {
}

const filePath = join(dir, file.fileName);
await fs.promises.writeFile(filePath, source, 'utf8');

const options = {
encoding: DEFAULT_FILE_ENCODING,
...fileStorageParams,
};

await fs.promises.writeFile(filePath, source, options);

return DataMapper.create<FileWriteResult>(FileWriteResult, {
md5: await md5File(filePath),
Expand Down
15 changes: 12 additions & 3 deletions src/domain/storages/MinioS3Storage.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import {toInteger as _toInteger} from 'lodash';
import {Readable} from 'stream';
import {toInteger as _toInteger} from 'lodash';
import * as Minio from 'minio';
import {DataMapper} from '@steroidsjs/nest/usecases/helpers/DataMapper';
import {normalizeBoolean} from '@steroidsjs/nest/infrastructure/decorators/fields/BooleanField';
Expand Down Expand Up @@ -73,16 +73,25 @@ export class MinioS3Storage implements IFileStorage {
});
}

public async write(file: IFileWritable, source: Readable | Buffer): Promise<FileWriteResult> {
public async write(
file: IFileWritable,
source: Readable | Buffer,
fileStorageParams: Record<string, any> | null = {},
): Promise<FileWriteResult> {
await this.makeMainBucket();

const metaData = {
'Content-Type': file.fileMimeType,
...fileStorageParams,
};

return new Promise((resolve, reject) => {
this.getClient().putObject(
this.mainBucket,
[file.folder, file.fileName].filter(Boolean).join('/'),
source,
file.fileSize,
{'Content-Type': file.fileMimeType},
metaData,
(err, {etag}) => {
if (err) {
reject(err);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export const GET_FILE_STORAGE_PARAMS_USE_CASE_TOKEN = 'get_file_storage_params_use_case_token';

/*
Позволяет задать параметры загрузки файла в определенный тип хранилища
*/
export interface IGetFileStorageParamsUseCase {
handle: (fileType: string, storageName: string) => Promise<Record<string, any>>,
}