Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EMA-4088 & EMA-4345]: Integrate Action Flow Import/Export functionality #190

Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@
- [List all packages in Studio](#list-all-packages-in-studio)
- [List assignments](#list-assignments)
- [Asset options for Analysis](#asset-options-for-analysis)
- [Action Flows commands](#action-flows-commands)
- [Analyze Action Flows](#analyze-action-flows)
- [Export Action Flows](#export-action-flows)
- [Import Action Flows](#import-action-flows)
- [Data Pool export / import commands](#data-pool-export--import-commands)
- [Export Data Pool](#export-data-pool)
- [Batch Import multiple Data Pools](#batch-import-multiple-data-pools)
Expand Down Expand Up @@ -440,6 +444,59 @@ the same command as with pushing other assets to Studio:
content-cli push bookmarks -p my-profile-name --id 73d39112-73ae-4bbe-8051-3c0f14e065ec --file studio_analysis_bookmarks_39c5bb7b-b486-4230-ab01-854a17ddbff2.json
```

### Action Flows commands

#### Analyze Action Flows

The analyze operation returns the metadata of Action Flows for one package together with their dependencies. Dependencies
could be webhooks, data structures, variables and other.

In order to analyze Action Flows you can execute the following command:

```
content-cli analyze action-flows -p my-profile-name --packageId <replace-with-package-id> --outputToJsonFile
```

The ```--outputToJsonFile``` is optional. If specified, the analyze result is saved in a JSON file (```metadata.json```). The
command output will give you all the details.

#### Export Action Flows

The export operation allows export of Action Flows of one package together with their dependencies. Dependencies
could be webhooks, data structures, variables and other.

In order to pull Action Flows you can execute the following command:

```
content-cli export action-flows -p my-profile-name --packageId <replace-with-package-id> -f <replace-with-metadata-file-name>
```

_Note_: The ```-f``` is optional. If specified, it will attach the metadata file to the exported zip.
This file is expected to be received by the ```action-flows analyze``` command, and manually be populated with the mappings source to target package.

#### Import Action Flows

The import operation allows import of Action Flows of one package together with their dependencies. Dependencies
could be webhooks, data structures, variables and other.

In order to push Action Flows use the following command:

```
content-cli import action-flows -p my-profile-name --packageId <replace-with-package-id> -f <replace-with-exported-zip-file> --dryRun --outputToJsonFile
```

##### Input

* The zip file is the one that you receive from the ```action-flows export``` command
* The ```--outputToJsonFile``` is optional. If specified, the import will be executed on dry run mode (nothing will be created)
jetakasabaqi marked this conversation as resolved.
Show resolved Hide resolved
* This is expected to be used to test the import by looking at report that is received on the response
* The ```--outputToJsonFile``` is optional. If specified, the import result is saved in a JSON file. The
command output will give you all the details.

#### Output

The command outputs an import report (event log).

### Data Pool export / import commands

#### Export Data Pool
Expand Down
31 changes: 31 additions & 0 deletions src/api/action-flow-api.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { httpClientV2 } from "../services/http-client-service.v2";
import { FatalError } from "../util/logger";
import * as FormData from "form-data";

class ActionFlowApi {
public static readonly INSTANCE = new ActionFlowApi();

public async exportRawAssets(packageId: string): Promise<Buffer> {
return httpClientV2.getFile(`/ems-automation/api/root/${packageId}/export/assets`).catch(e => {
throw new FatalError(`Problem getting Action Flow assets: ${e}`);
});
}

public async analyzeAssets(packageId: string): Promise<any> {
return httpClientV2.get(`/ems-automation/api/root/${packageId}/export/assets/analyze`).catch(e => {
throw new FatalError(`Problem analyzing Action Flow assets: ${e}`);
});
}

public async importAssets(packageId: string, data: FormData, dryRun: boolean): Promise<any> {
const params = {
dryRun: dryRun,
};

return httpClientV2.postFile(`/ems-automation/api/root/${packageId}/import/assets`, data, params).catch(e => {
throw new FatalError(`Problem importing Action Flow assets: ${e}`);
});
}
}

export const actionFlowApi = ActionFlowApi.INSTANCE;
16 changes: 16 additions & 0 deletions src/commands/action-flow.command.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { actionFlowService } from "../services/action-flow/action-flow-service";

export class ActionFlowCommand {

public async exportActionFlows(packageId: string, metadataFile: string): Promise<void> {
await actionFlowService.exportActionFlows(packageId, metadataFile);
}

public async analyzeActionFlows(packageId: string, outputToJsonFile: boolean): Promise<void> {
await actionFlowService.analyzeActionFlows(packageId, outputToJsonFile);
}

public async importActionFlows(packageId: string, filePath: string, dryRun: boolean, outputToJsonFile: boolean): Promise<void> {
await actionFlowService.importActionFlows(packageId, filePath, dryRun, outputToJsonFile);
}
}
43 changes: 43 additions & 0 deletions src/content-cli-analyze.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import { ContextInitializer } from "./util/context-initializer";
import { logger } from "./util/logger";
import { ActionFlowCommand } from "./commands/action-flow.command";
import commander = require("commander");

type CommanderStatic = commander.CommanderStatic;

class Analyze {
public static actionFlows(program: CommanderStatic): CommanderStatic {
program
.command("action-flows")
.description("Analyze Action Flows dependencies for a certain package")
.option("-p, --profile <profile>", "Profile which you want to use to analyze Action Flows")
.requiredOption("--packageId <packageId>", "ID of the package from which you want to export Action Flows")
.option("-o, --outputToJsonFile", "Output the analyze result in a JSON file")
promeris marked this conversation as resolved.
Show resolved Hide resolved
.action(async cmd => {
await new ActionFlowCommand().analyzeActionFlows(cmd.packageId, cmd.outputToJsonFile);
process.exit();
});
return program;
}
}

const loadCommands = () => {
getAllCommands();
};

ContextInitializer.initContext()
.then(loadCommands, loadCommands)
.catch(e => {
logger.error(e);
});

if (!process.argv.slice(2).length) {
commander.outputHelp();
process.exit(1);
}

function getAllCommands(): void {
Analyze.actionFlows(commander);

commander.parse(process.argv);
}
17 changes: 17 additions & 0 deletions src/content-cli-export.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { PackageCommand } from "./commands/package.command";
import { logger } from "./util/logger";
import { DataPoolCommand } from "./commands/data-pool.command";
import { ContextInitializer } from "./util/context-initializer";
import { ActionFlowCommand } from "./commands/action-flow.command";

export class Export {
public static packages(program: CommanderStatic): CommanderStatic {
Expand Down Expand Up @@ -36,6 +37,21 @@ export class Export {

return program;
}

public static actionFlows(program: CommanderStatic): CommanderStatic {
program
.command("action-flows")
.description("Command to export all Action Flows in a package with their objects and dependencies")
.option("-p, --profile <profile>", "Profile which you want to use to export Action Flows")
.requiredOption("--packageId <packageId>", "ID of the package from which you want to export Action Flows")
.option("-f, --file <file>", "Action flows metadata file (relative path)")
.action(async cmd => {
await new ActionFlowCommand().exportActionFlows(cmd.packageId, cmd.file);
process.exit();
});

return program;
}
}

const loadCommands = () => {
Expand All @@ -56,6 +72,7 @@ if (!process.argv.slice(2).length) {
function getAllCommands(): void {
Export.packages(commander);
Export.dataPool(commander);
Export.actionFlows(commander);

commander.parse(process.argv);
}
19 changes: 19 additions & 0 deletions src/content-cli-import.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { PackageCommand } from "./commands/package.command";
import { DataPoolCommand } from "./commands/data-pool.command";
import { ContextInitializer } from "./util/context-initializer";
import { logger } from "./util/logger";
import { ActionFlowCommand } from "./commands/action-flow.command";

export class Import {
public static packages(program: CommanderStatic): CommanderStatic {
Expand Down Expand Up @@ -41,6 +42,23 @@ export class Import {

return program;
}

public static actionFlows(program: CommanderStatic): CommanderStatic {
program
.command("action-flows")
.description("Command to import all Action Flows in a package with their objects and dependencies")
.option("-p, --profile <profile>", "Profile which you want to use to import Action Flows")
.requiredOption("--packageId <packageId>", "ID of the package to which you want to export Action Flows")
.requiredOption("-f, --file <file>", "Exported Action Flows file (relative path)")
.requiredOption("-d, --dryRun <dryRun>", "Execute the import on dry run mode")
.option("-o, --outputToJsonFile", "Output the import result in a JSON file")
.action(async cmd => {
await new ActionFlowCommand().importActionFlows(cmd.packageId, cmd.file, cmd.dryRun, cmd.outputToJsonFile);
process.exit();
});

return program;
}
}

const loadCommands = () => {
Expand All @@ -61,6 +79,7 @@ if (!process.argv.slice(2).length) {
function getAllCommands(): void {
Import.packages(commander);
Import.dataPools(commander);
Import.actionFlows(commander);

commander.parse(process.argv);
}
4 changes: 3 additions & 1 deletion src/content-cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ program.command("profile", "Commands related to profiles.");
program.command("pull", "Commands to pull content.");

program.command("export", "Commands to export content.")
program.command("import", "Commands to export content.")
program.command("import", "Commands to import content.")

program.command("push", "Commands to push content.");

Expand All @@ -32,6 +32,8 @@ program.command("set", "Commands to set configuration properties.");

program.command("config", "Commands related to config management.")

program.command("analyze", "Commands to analyze assets dependencies.");

program.version(VersionUtils.getCurrentCliVersion());
program.parse(process.argv);

Expand Down
74 changes: 74 additions & 0 deletions src/services/action-flow/action-flow-service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import { logger } from "../../util/logger";
import { v4 as uuidv4 } from "uuid";
import { FileService, fileService } from "../file-service";
import { actionFlowApi } from "../../api/action-flow-api";
import * as AdmZip from "adm-zip";
import * as FormData from "form-data";
import * as fs from "fs";

class ActionFlowService {
private static readonly METADATA_FILE_NAME = "metadata.json";

public async exportActionFlows(packageId: string, metadataFilePath: string): Promise<void> {
const exportedActionFlowsData = await actionFlowApi.exportRawAssets(packageId);
const tmpZip: AdmZip = new AdmZip(exportedActionFlowsData);

const zip = new AdmZip();
tmpZip.getEntries().forEach(entry => {
jetakasabaqi marked this conversation as resolved.
Show resolved Hide resolved
zip.addFile(entry.entryName, entry.getData());
});

if (metadataFilePath) {
this.attachMetadataFile(metadataFilePath, zip);
}

const fileName = "action-flows_export_" + uuidv4() + ".zip";
zip.writeZip(fileName);
logger.info(FileService.fileDownloadedMessage + fileName);
}

public async analyzeActionFlows(packageId: string, outputToJsonFile: boolean): Promise<void> {
const actionFlowsMetadata = await actionFlowApi.analyzeAssets(packageId);
const actionFlowsMetadataString = JSON.stringify(actionFlowsMetadata, null, 4);

if (outputToJsonFile) {
const metadataFileName = "action-flows_metadata_" + uuidv4() + ".json";
fileService.writeToFileWithGivenName(actionFlowsMetadataString, metadataFileName);
logger.info(FileService.fileDownloadedMessage + metadataFileName);
} else {
logger.info("Action flows analyze metadata: \n" + actionFlowsMetadataString);
}
}

public async importActionFlows(packageId: string, filePath: string, dryRun: boolean, outputToJsonFile: boolean): Promise<void> {
const actionFlowsZip = this.createBodyForImport(filePath);
const eventLog = await actionFlowApi.importAssets(packageId, actionFlowsZip, dryRun);
const eventLogString = JSON.stringify(eventLog, null, 4);

if (outputToJsonFile) {
const eventLogFileName = "action-flows_import_event_log_" + uuidv4() + ".json";
fileService.writeToFileWithGivenName(eventLogString, eventLogFileName);
logger.info(FileService.fileDownloadedMessage + eventLogFileName);
} else {
logger.info("Action flows import event log: \n" + eventLogString);
}
}

private createBodyForImport(fileName: string): FormData {
fileName = fileName + (fileName.endsWith(".zip") ? "" : ".zip");

const formData = new FormData();
formData.append("file", fs.createReadStream(fileName, { encoding: null }), { filename: fileName });

return formData;
}

private attachMetadataFile(fileName: string, zip: AdmZip): void {
fileName = fileName + (fileName.endsWith(".json") ? "" : ".json");
const metadata = fileService.readFile(fileName);

zip.addFile(ActionFlowService.METADATA_FILE_NAME, Buffer.from(metadata));
}
}

export const actionFlowService = new ActionFlowService();
Loading