coypu

CoyPu Crisis-LM

Data

When running python main.py the CrisisBench dataset is automatically downloaded to a data folder within this project (if you don’t already have it there). From this, a huggingface dataset is created.

Fine-tuning approaches

This repository will experiment with different domain-/task-adapting fine-tuning approaches.

• Task-adaptive fine-tuning of BERTweet on unsupervised MLM objective
• Adapter-based fine-tuning
• Additional tuning towards SentenceTransformer embeddings

Evaluation

Evaluation of the embeddings is done via

• MLM perplexity
• Downstream task performance on the CrisisBench benchmark
• Additional CoyPu-specific downstream tasks: e.g., event detection

Running the MLM demo

Get the CoyPu CrisisLM model version 1 here. Unzip and provide path as described below.

Configure as follows: /conf/mode/mlm_inference.yaml: inference_mode: "interactive" /conf/model/bertweet.yaml: pretrained_model: PATH TO MODEL FOLDER /conf/config.yaml: defaults: - mode: mlm_inference - model: bertweet

Then simply run python main.py.

This site is open source. Improve this page.