## An API standard for multi-objective multi-agent reinforcement learning (MOMARL)

MOMAland is an open source Python library for developing and comparing multi-objective multi-agent reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. Essentially, the environments follow the standard PettingZoo APIs, but return vectorized rewards as numpy arrays instead of scalar values.

The documentation website is at https://momaland.farama.org/, and we have a public discord server (which we also use to coordinate development work) that you can join here.

# API¶

Similar to PettingZoo, the MOMAland API models environments as simple Python `env`

classes. Creating environment instances and interacting with them is very simple - here’s an example using the “momultiwalker_stability_v0” environment:

```
from momaland.envs.momultiwalker_stability import momultiwalker_stability_v0 as _env
import numpy as np
# .env() function will return an AEC environment, as per PZ standard
env = _env.env(render_mode="human")
env.reset(seed=42)
for agent in env.agent_iter():
# vec_reward is a numpy array
observation, vec_reward, termination, truncation, info = env.last()
if termination or truncation:
action = None
else:
action = env.action_space(agent).sample() # this is where you would insert your policy
env.step(action)
env.close()
# optionally, you can scalarize the reward with weights
# Making the vector reward a scalar reward to shift to single-objective multi-agent (aka PettingZoo)
# We can assign different weights to the objectives of each agent.
weights = {
"walker_0": np.array([0.7, 0.3]),
"walker_1": np.array([0.5, 0.5]),
"walker_2": np.array([0.2, 0.8]),
}
env = LinearizeReward(env, weights)
```

For details on multi-objective multi-agent RL definitions, see Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey.

# Install¶

To install MOMAland, use:

```
pip install momaland
```

This does not include dependencies for all components of MOMAland (not everything is required for the basic usage, and some can be problematic to install on certain systems).

`pip install "momaland[testing]"`

to install dependencies for API testing.`pip install "momaland[learning]"`

to install dependencies for the supplied learning algorithms.`pip install "momaland[all]"`

for all dependencies for all components.

# Citing¶

If you use this repository in your research, please cite:

```
@misc{felten2024momaland,
title={MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning},
author={Florian Felten and Umut Ucak and Hicham Azmani and Gao Peng and Willem Röpke and Hendrik Baier and Patrick Mannion and Diederik M. Roijers and Jordan K. Terry and El-Ghazali Talbi and Grégoire Danoy and Ann Nowé and Roxana Rădulescu},
year={2024},
eprint={2407.16312},
archivePrefix={arXiv},
primaryClass={cs.MA},
url={https://arxiv.org/abs/2407.16312},
}
```