---
license: apache-2.0
base_model: swiss-ai/Apertus-70B-Instruct-2509
pipeline_tag: text-generation
library_name: node-llama-cpp
tags:
- node-llama-cpp
- llama.cpp
- apertus
- multilingual
- swiss-ai
- compliant
- conversational
extra_gated_prompt: "### Apertus LLM Acceptable Use Policy  \n(1.0 | September 1,\
  \ 2025)\n\"Agreement\" The Swiss National AI Institute (SNAI) is a partnership between\
  \ the two Swiss Federal Institutes of Technology, ETH Zurich and EPFL. \n\nBy using\
  \ the Apertus LLM you agree to indemnify, defend, and hold harmless ETH Zurich and\
  \ EPFL against any third-party claims arising from your use of Apertus LLM. \n\n\
  The training data and the Apertus LLM may contain or generate information that directly\
  \ or indirectly refers to an identifiable individual (Personal Data). You process\
  \ Personal Data as independent controller in accordance with applicable data protection\
  \ law. SNAI will regularly provide a file with hash values for download which you\
  \ can apply as an output filter to your use of our Apertus LLM. The file reflects\
  \ data protection deletion requests which have been addressed to SNAI as the developer\
  \ of the Apertus LLM. It allows you to remove Personal Data contained in the model\
  \ output. We strongly advise downloading and applying this output filter from SNAI\
  \ every six months following the release of the model.  "
extra_gated_fields:
  Your Name: text
  Country: country
  Affiliation: text
  geo: ip_location
  By clicking Submit below I accept the terms of use: checkbox
extra_gated_button_content: Submit
quantized_by: giladgd
---

# Apertus-70B-Instruct-2509-GGUF

Static quants of [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509).


## Quants
| Link | [URI](https://node-llama-cpp.withcat.ai/cli/pull) | Quant | Size |
|:-----|:--------------------------------------------------|:------|-----:|
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q2_K.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q2_K` | Q2_K | 27.3GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q3_K_S.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q3_K_S` | Q3_K_S | 30.8GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q3_K_M.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q3_K_M` | Q3_K_M | 35.5GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q3_K_L.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q3_K_L` | Q3_K_L | 39.6GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q4_0.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_0` | Q4_0 | 40.0GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q4_K_S.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_S` | Q4_K_S | 40.4GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q4_K_M.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M` | Q4_K_M | 43.7GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q5_0.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q5_0` | Q5_0 | 48.7GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q5_K_S.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q5_K_S` | Q5_K_S | 48.7GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q5_K_M.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q5_K_M` | Q5_K_M | 50.6GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q6_K.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q6_K` | Q6_K | 57.9GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q8_0.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q8_0` | Q8_0 | 75.0GB |
| [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.F16.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:F16` | F16 | 141.2GB |

> [!TIP]
> Download a quant using `node-llama-cpp` ([more info](https://node-llama-cpp.withcat.ai/cli/pull)):
> ```bash
> npx -y node-llama-cpp pull <URI>
> ```


# Usage
## Use with [`node-llama-cpp`](https://node-llama-cpp.withcat.ai) (recommended)
Ensure you have node.js installed:
```bash
brew install nodejs
```

### CLI
Chat with the model:
```bash
npx -y node-llama-cpp chat hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M
```

### Code
Use it in your project:
```bash
npm install node-llama-cpp
```

```typescript
import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";

const modelUri = "hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M";


const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: await resolveModelFile(modelUri)
});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});


const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);
```

> [!TIP]
> Read the [getting started guide](https://node-llama-cpp.withcat.ai/guide/) to quickly scaffold a new `node-llama-cpp` project


## Use with [llama.cpp](https://github.com/ggml-org/llama.cpp)
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp
```

### CLI
```bash
llama-cli -hf giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M -p "The meaning to life and the universe is"
```

### Server
```bash
llama-server -hf giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M -c 2048
```