YANS 2019: Annotation Hackathon

• YANS 2019

Annotation Workflow

Detail

Annotator Workflow

• You can easily deploy doccano by Heroku Button.
  • https://github.com/chakki-works/doccano
  • Recommendation: Use above hobby (for Heroku)/ hobby-basic (PostgreSQL) plan.

Evaluation Backend

• S3 bucket: yans.2019.js
• Google book id: 1WDwojAFoswN_rBe0P31sKECcWeku25fgLtAG7ZSbAUo
• Lambda function is triggerd by S3 upload. Lambda implementation is defined at yans/function/lambda_function.py.

Resouces

• Slack => S3: Integromat
• Run Training: AWS
  • Use ECS to run Dockerfile
• Japanese NER model is based on GiNZA.

Scripts

• Prepare data
  • Download and preprocess: python yans/data/prepare.py
• Train by data
  • python yans/data/train.py
  • If --local is set, use local data directory and else use S3.
• Evaluate model
  • python yans/data/evaluate.py
  • If --local is set, use local data directory and else use S3.
• Rule based annotation
  • python yans/annotation/annotate.py

Result

• Google Data studio Dashboard
  • Data source Spread Sheet