{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Anti-epanorthosis LoRA adapter — Google Colab (free T4)\n",
    "\n",
    "Companion to *Artificial Epanorthosis* (§7.2, Appendix A). Trains a small, detachable **LoRA** adapter with **DPO** to prefer affirmative phrasing over gratuitous negate-and-upgrade epanorthosis, at fixed content.\n",
    "\n",
    "**Before running:** menu **Runtime → Change runtime type → T4 GPU**. Then run the cells top to bottom.\n",
    "\n",
    "The base model here is small and multilingual (Qwen2.5-1.5B-Instruct) so it fits the free T4. On a bigger GPU bump `BASE_MODEL` to a 7–8B instruct model. This is a **pipeline demonstration**: 22 seed pairs prove the loop end-to-end; for a strong adapter, scale the dataset to hundreds–thousands of clean pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi -L   # confirm a GPU is attached (expect a Tesla T4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip -q install \"transformers>=4.44\" \"trl>=0.11\" \"peft>=0.13\" \"datasets>=2.20\" \"accelerate>=0.34\" \"bitsandbytes>=0.43\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Preference data\n",
    "Upload `artificial-epanorthosis-pairs.jsonl` (from the paper's supplementary files) when prompted. If you skip the upload, a tiny built-in fallback set is used so the notebook still runs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json, os\n",
    "from datasets import Dataset\n",
    "\n",
    "FALLBACK = [\n",
    "  {\"prompt\": \"Scrivi due frasi motivazionali per un post.\", \"chosen\": \"Il lavoro da remoto dà alle persone il controllo del proprio tempo. Chi lo adotta trattiene i talenti migliori.\", \"rejected\": \"Il lavoro da remoto non è un rischio, è un'opportunità. Non perdi controllo: anzi, trattieni i talenti migliori.\"},\n",
    "  {\"prompt\": \"Presenta il tuo lavoro in una frase.\", \"chosen\": \"Progetto percorsi di formazione pratici sull'intelligenza artificiale.\", \"rejected\": \"Non vendo corsi. Costruisco percorsi di trasformazione.\"},\n",
    "  {\"prompt\": \"Chiudi un discorso di diploma in una frase.\", \"chosen\": \"Il diploma è un lasciapassare: usatelo per continuare a imparare.\", \"rejected\": \"Il diploma non è un punto di arrivo, è un punto di partenza.\"}\n",
    "]\n",
    "\n",
    "path = None\n",
    "try:\n",
    "    from google.colab import files\n",
    "    print('Upload artificial-epanorthosis-pairs.jsonl (or press Cancel to use the fallback set)')\n",
    "    up = files.upload()\n",
    "    if up:\n",
    "        path = list(up.keys())[0]\n",
    "except Exception:\n",
    "    pass\n",
    "\n",
    "rows = []\n",
    "if path and os.path.exists(path):\n",
    "    with open(path) as f:\n",
    "        rows = [json.loads(l) for l in f if l.strip()]\n",
    "if not rows:\n",
    "    rows = FALLBACK\n",
    "print('preference pairs:', len(rows))\n",
    "raw = Dataset.from_list(rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Base model (4-bit / QLoRA)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n",
    "\n",
    "BASE_MODEL = \"Qwen/Qwen2.5-1.5B-Instruct\"   # ungated, multilingual, fits a free T4\n",
    "\n",
    "bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n",
    "                         bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)\n",
    "tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)\n",
    "if tokenizer.pad_token is None:\n",
    "    tokenizer.pad_token = tokenizer.eos_token\n",
    "model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, quantization_config=bnb, device_map=\"auto\")\n",
    "model.config.use_cache = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Format prompts with the chat template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fmt(ex):\n",
    "    ex[\"prompt\"] = tokenizer.apply_chat_template(\n",
    "        [{\"role\": \"user\", \"content\": ex[\"prompt\"]}], tokenize=False, add_generation_prompt=True)\n",
    "    return ex\n",
    "\n",
    "dataset = raw.map(fmt)\n",
    "print(dataset[0][\"prompt\"][:200])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Train the LoRA adapter with DPO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from peft import LoraConfig\n",
    "from trl import DPOTrainer, DPOConfig\n",
    "\n",
    "peft_config = LoraConfig(\n",
    "    r=16, lora_alpha=32, lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\",\n",
    "    target_modules=[\"q_proj\", \"v_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"])\n",
    "\n",
    "config = DPOConfig(\n",
    "    output_dir=\"epanorthosis-deemphasis-lora\",\n",
    "    beta=0.1, learning_rate=5e-5, num_train_epochs=3,\n",
    "    per_device_train_batch_size=1, gradient_accumulation_steps=8,\n",
    "    max_length=1024, max_prompt_length=512,\n",
    "    lr_scheduler_type=\"cosine\", warmup_ratio=0.1,\n",
    "    logging_steps=5, save_strategy=\"no\", fp16=True, report_to=\"none\")\n",
    "\n",
    "trainer = DPOTrainer(model=model, args=config, train_dataset=dataset,\n",
    "                     processing_class=tokenizer, peft_config=peft_config)\n",
    "trainer.train()\n",
    "trainer.save_model(\"epanorthosis-deemphasis-lora\")\n",
    "print(\"adapter saved to epanorthosis-deemphasis-lora/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Evaluate: base vs. adapter, with the epanorthosis detector\n",
    "Generate on held-out Italian prompts and count the emphatic-correction family («non X ma/bensì Y», «non solo X ma Y», «non X, è Y», and the split «Non è X. È Y») per 10k words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "from peft import PeftModel\n",
    "\n",
    "def words(t): return re.findall(r\"[A-Za-zàèéìòù']+\", t)\n",
    "RE_NONMA=re.compile(r\"\\bnon\\b(?:(?!\\bma\\b|\\bbens[iì]\\b)[^.;:!?\\n]){2,70}?\\b(ma|bens[iì])\\b\", re.I)\n",
    "RE_NONSOLO=re.compile(r\"\\bnon solo\\b(?:[^.;:!?\\n]){2,90}?\\bma\\b\", re.I)\n",
    "RE_NONVIRG=re.compile(r\"\\bnon\\b(?:(?!\\bma\\b)[^.;:!?\\n]){2,55}?,\\s*(è|e'|sono|era|significa)\\b\", re.I)\n",
    "NEG=re.compile(r\"^\\W*(non è|non sono|non ho|non facc|non vend|questa? non è|non si tratta di)\\b\", re.I)\n",
    "IDIT=re.compile(r\"non solo|non appena|non che\\b\", re.I)\n",
    "def emph(t):\n",
    "    nm=[m.group(0) for m in RE_NONMA.finditer(t) if not IDIT.search(m.group(0))]\n",
    "    ss=re.split(r\"(?<=[.!?])\\s+\", t); nx=0\n",
    "    for i in range(len(ss)-1):\n",
    "        if NEG.match(ss[i].strip()) and not NEG.match(ss[i+1].strip()): nx+=1\n",
    "    return len(nm)+len(RE_NONSOLO.findall(t))+nx+len(RE_NONVIRG.findall(t))\n",
    "def density(t): n=len(words(t)); return round(emph(t)*10000/n,1) if n else 0\n",
    "\n",
    "PROMPTS = [\n",
    "  \"Scrivi un breve discorso motivazionale per l'inaugurazione di una scuola.\",\n",
    "  \"Scrivi un post su LinkedIn sul lancio della tua attività di consulenza.\",\n",
    "  \"Scrivi un saggio breve sul perché la lettura conta ancora.\",\n",
    "]\n",
    "\n",
    "def generate(m, prompt):\n",
    "    msgs=[{\"role\":\"user\",\"content\":prompt}]\n",
    "    ids=tokenizer.apply_chat_template(msgs, add_generation_prompt=True, return_tensors=\"pt\").to(m.device)\n",
    "    out=m.generate(ids, max_new_tokens=320, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.pad_token_id)\n",
    "    return tokenizer.decode(out[0][ids.shape[1]:], skip_special_tokens=True)\n",
    "\n",
    "tuned = PeftModel.from_pretrained(model, \"epanorthosis-deemphasis-lora\")\n",
    "for p in PROMPTS:\n",
    "    tuned.disable_adapter_layers(); base_txt = generate(tuned, p)\n",
    "    tuned.enable_adapter_layers();  lora_txt = generate(tuned, p)\n",
    "    print(\"\\n=== PROMPT:\", p)\n",
    "    print(f\"  base  density={density(base_txt):5}  | {base_txt[:110]!r}\")\n",
    "    print(f\"  lora  density={density(lora_txt):5}  | {lora_txt[:110]!r}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reading the result.** Success is a lower `density` for `lora` than `base` *while the content stays intact*. Compare the adapter's density to the human baseline for the genre (paper §7.7): the target is to match the human rate, not to drive it to zero. With only 22 seed pairs expect a modest shift; scale the dataset (more genres/topics/languages, filtered so `chosen` truly has lower density) for a strong adapter.\n",
    "\n",
    "To keep the adapter: `!zip -r adapter.zip epanorthosis-deemphasis-lora` then download it from the Files panel."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {"provenance": [], "gpuType": "T4"},
  "kernelspec": {"display_name": "Python 3", "name": "python3"},
  "language_info": {"name": "python"}
 },
 "nbformat": 4,
 "nbformat_minor": 0
}