README.md

# FunctionGemma Infrastructure LiteRT-LM

A fine-tuned [FunctionGemma-270M](https://huggingface.co/google/functiongemma-270m-it) model converted to LiteRT-LM format for on-device inference. Designed for self-healing infrastructure and automatic error remediation.

## Model Details

| Property | Value |
|----------|-------|
| Base Model | google/functiongemma-270m-it |
| Format | LiteRT-LM (.litertlm) |
| Quantization | Dynamic INT8 |
| File Size | 272 MB |
| Parameters | 270M |

## Intended Use

This model is designed for [dad-express](https://github.com/anthropics/dad-express), a self-healing gateway that monitors HTTP traffic and automatically fixes infrastructure configuration issues by calling the appropriate tools.

## Supported Tools

The model was fine-tuned on 9 infrastructure tools:

| Tool | Description | Parameters |
|------|-------------|------------|
| `addProxyRoute` | Add reverse proxy route | path, upstream, port |
| `addCorsHeaders` | Configure CORS headers | origin, credentials |
| `configureSsl` | Configure SSL certificate | hostname, selfSigned |
| `setEnvVariable` | Set environment variable | name, value |
| `exposePort` | Expose port in Docker/firewall | service, port |
| `addHostEntry` | Add hostname to /etc/hosts | hostname, ip |
| `restartService` | Restart a service | service |
| `clearCache` | Clear cache | cacheType |
| `modifyConfig` | Modify config file | file, key, value |

## Training Details

### Dataset
- **Total Examples**: 10,216
- **Train/Eval Split**: 90/10 (9,194 train, 1,022 eval)
- **Format**: Prompt-completion pairs using FunctionGemma chat template
- **Distribution**: ~1,000-1,200 examples per tool (balanced)

### Training Configuration

Trained following [Google's official FunctionGemma fine-tuning notebook](https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/FunctionGemma/%5BFunctionGemma%5DFinetune_FunctionGemma_270M_for_Mobile_Actions_with_Hugging_Face.ipynb):

```python
SFTConfig(
    num_train_epochs=2,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=1e-5,
    lr_scheduler_type="cosine",
    gradient_checkpointing=True,
    packing=False,
    optim="adamw_torch_fused",
    bf16=True,
    completion_only_loss=True,  # Critical: only train on completion tokens
)
```

### Hardware
- **GPU**: NVIDIA L4 (24GB)
- **Training Time**: ~23 minutes
- **Conversion Time**: ~10 minutes

### Performance Metrics

| Metric | Value |
|--------|-------|
| Final Eval Loss | 0.034 |
| Token Accuracy | 98.6% |
| Training Steps | 576 |

## LiteRT-LM Conversion

Converted using [ai-edge-torch](https://github.com/google-ai-edge/ai-edge-torch):

```python
converter.convert_to_litert(
    pytorch_model,
    prefill_seq_len=256,
    kv_cache_max_len=1024,
    quantize="dynamic_int8",
    output_format="litertlm",
)
```

### LLM Metadata
```protobuf
start_token: { token_ids: { ids: [ 2 ] } }
stop_tokens: { token_str: "<end_of_turn>" }
stop_tokens: { token_str: "<start_function_response>" }
llm_model_type: { function_gemma: {} }
```

## Usage

### With LiteRT-LM Runtime
```typescript
import { LiteRTLM } from 'litert-lm';

const model = await LiteRTLM.load('functiongemma-infrastructure_q8_ekv1024.litertlm');
const response = await model.generate(prompt);
```

### Example Input/Output

**Input:**
```
Error: CORS - No 'Access-Control-Allow-Origin' header from http://localhost:3000
```

**Output:**
```
<start_function_call>call:addCorsHeaders{origin:<escape>http://localhost:3000<escape>,credentials:<escape>true<escape>}<end_function_call><start_function_response>
```

## Dependencies

- Python 3.11+
- transformers==4.57.1
- trl==0.25.1
- datasets==4.4.1
- ai-edge-torch-nightly
- ai-edge-litert-nightly

## License

This model inherits the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).

## Citation

```bibtex
@misc{functiongemma-infrastructure-litertlm,
  title={FunctionGemma Infrastructure LiteRT-LM},
  author={dad-express contributors},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/macmacmacmac/functiongemma-infrastructure-litertlm}
}
```