# Executing LLM-generated code We have built a code-executor that can be used to execute LLM-generated code. This is useful for many situations: 1. You want to include error-free code in your training code. This method is used in [Open Thoughts](https://open-thoughts.ai). 2. LLM generates some code to generate visualization etc. 3. Agents and tool-use. Here is a simple example of code execution in action: ```python from bespokelabs import curator from datasets import Dataset class HelloExecutor(curator.CodeExecutor): def code(self, row): return """location = input();print(f"Hello {location}")""" def code_input(self, row): return row['location'] def code_output(self, row, execution_output): row['output'] = execution_output.stdout return row locations = Dataset.from_list([{'location': 'New York'},{'location': 'Tokyo'}]) hello_executor = HelloExecutor() print(hello_executor(locations).to_pandas()) ``` The inherited class contains three methods: 1. `code`: This is the method that returns the piece of code to be run. This is usually part of the row (you can use `curator.LLM` to generate this code). 2. `code_input`: This is optional, but can return a json that represents values to be passed to `input()` in the code. 3. `code_output`: This is where you parse the output of the execution. ### Features: 1. Full caching and automatic recovery: Similar to [Curator.LLM's caching feature](https://docs.bespokelabs.ai/bespoke-curator/tutorials/automatic-recovery-and-caching), code executor also has inbuilt caching and automatic recovery. Any interrupted runs can be fully recovered and no computation is lost. 2. Multiple code execution backends: We offer **four** backends: multiprocessing, docker, ray and E2B. These backends specify different locations where your code can be executed. You can easily switch the backend with a simple parameter change. For example, the hello world example can be run using the ray backend by simply initializing it with \``HelloExecutor(backend=ray)`\` 3. Progress monitoring using Rich Console:
### Backends We offer **four** backends for running your code: 1. Multiprocessing: This is the default backend. This runs code locally and is therefore the least safe option, but is useful for quick execution as it does not require any dependencies. 2. Docker: It is safer option than multiprocessing. 3. Ray: If you have a ray cluster, you can use it by setting `CodeExecutor(backend="ray")`. This is useful when your code can take a long time to run. 4. E2B: Code can also be run using [e2b.dev](https://e2b.dev/). Use `CodeExecutor(backend="e2b")` . ### Backend Setup and configuration options #### Multiprocessing Backend: This doesn't require any additional setup. You can configure `backend params` while initializing as follows: 
```python
hello_executor = HelloExecutor(
    backend_params = {    
        "max_requests_per_minute": 1000
    }
)
```
You can also configure execution parameters: ```` ```python output = hello_executor(dataset, execution_params = { "timeout": 120 # in seconds "memory_limit": 1024 ** 3 } ) ``` ```` #### Docker With docker, code can be executed in a secure containerized environment. You need docker installed and python's docker client installed on your machine: {% tabs %} {% tab title="Mac" %} 1. pip install docker 2. Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) 3. In your terminal, run \`docker pull python:3.11-slim\` 4. Run the HelloExecutor example with HelloExecutor(backend=docker) {% endtab %} {% tab title="Linux" %} 1. pip install docker 2. Install [docker desktop](https://docs.docker.com/desktop/setup/install/linux/) (recommended) or optionally just install the docker engine 3. In your terminal, run \`docker pull python:3.11-slim\` 4. Run the HelloExecutor example with HelloExecutor(backend=docker) {% endtab %} {% endtabs %} With docker, you can specify a custom docker image to execute your code snippets: 
```python
hello_executor = HelloExecutor(
    backend = "docker",
    backend_params = {    
        "image": "andgineer/matplotlib"
    }
)
```
#### Ray As the size of the dataset grows, it becomes harder to scale code execution requirements on a single machine. In such scenarios, one can use the ray backend. Simply run `pip install ray` to install the dependencies required for ray backend. You need to separately spin up a ray cluster and enter the base\_url ([Installation instructions](https://docs.ray.io/en/latest/ray-core/starting-ray.html)). If base\_url is not entered, then a local ray cluster is spun up. 
```python
hello_executor = HelloExecutor(
    backend = "ray",
    backend_params = {    
        "base_url": "<url of ray cluster>"
    }
)
```
#### E2B We also add light support for e2b's hosted code execution backends. While not free, they are secure environments similar to docker environments and have more features. 1. Run `pip install e2b-code-interpreter` to install the required dependencies. 2. Create an account on e2b's website, get the API key and add it to your environment variables. 
```python
hello_executor = HelloExecutor(
    backend = "e2b",
)
```
### Conclusion Check out the [examples](https://github.com/bespokelabsai/curator/tree/main/examples/code-execution) to get started with code executor. If you have any questions, feel free to join our [Discord](https://discord.com/invite/KqpXvpzVBS) or [send us an email](mailto:company@bespokelabs.ai). --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.