## Frequently asked questions

(Send your questions as a patch to this FAQ! I'll review it, merge it, and
respond by commit with the answer.)

### Why did you start this project?

I wanted a next generation config management solution that didn't have all of
the design flaws or limitations that the current generation of tools do, and no
tool existed!

### Why did you choose `golang` for the project?

When I started working on the project, I needed to choose a language that
already had an implementation of a distributed consensus algorithm available.
That meant [Paxos](https://en.wikipedia.org/wiki/Paxos_(computer_science)) or
[Raft](https://en.wikipedia.org/wiki/Raft_(computer_science)). Golang was one
language that actually had two different Raft implementations, `etcd`, and
`consul`. Other design requirements included something that was reasonably fast,
typed and memory-safe, and suited for systems engineering. After a reasonably
extensive search, I chose `golang`. I think it was the right decision. There are
a number of other features of the language which helped influence the decision.

### How do I contribute to the project if I don't know `golang`?

There are many different ways you can contribute to the project. They can be
broadly divided into two main categories:

1. With contributions written in `golang`
2. With contributions _not_ written in `golang`

If you do not know `golang`, and have no desire to learn, you can still
contribute to mgmt by using it, testing it, writing docs, or even just by
telling your friends about it. If you don't mind some coding, learning about the
[mgmt language](https://purpleidea.com/blog/2018/02/05/mgmt-configuration-language/)
might be an enjoyable experience for you. It is a small [DSL](https://en.wikipedia.org/wiki/Domain-specific_language)
and not a general purpose programming language, and you might find it more fun
than what you're typically used to. One of the reasons the mgmt author got into
writing automation modules, was because he found it much more fun to build with
a higher level DSL, than in a general purpose programming language.

If you do not know `golang`, and would like to learn, are a beginner and want to
improve your skills, or want to gain some great interdisciplinary systems
engineering knowledge around a cool automation project, we're happy to mentor
you. Here are some pre-requisites steps which we recommend:

1. Make sure you have a somewhat recent GNU/Linux environment to hack on. A
recent [Fedora](https://getfedora.org/) or [Debian](https://www.debian.org/)
environment is recommended. Developing, testing, and contributing on `macOS` or
`Windows` will be either more difficult or impossible.
2. Ensure that you're mildly comfortable with the basics of using `git`. You can
find a number of tutorials online.
3. Spend between four to six hours with the [golang tour](https://tour.golang.org/).
Skip over the longer problems, but try and get a solid overview of everything.
If you forget something, you can always go back and repeat those parts.
4. Connect to our [#mgmtconfig](https://matrix.to/#/#mgmtconfig:matrix.org)
Matrix channel and hang out with us there out there.
5. Now it's time to try and starting writing a patch! We have tagged a bunch of
[open issues as #mgmtlove](https://github.com/purpleidea/mgmt/issues?q=is%3Aissue+is%3Aopen+label%3Amgmtlove)
for new users to have somewhere to get involved. Look through them to see if
something interests you. If you find one, let us know you're working on it by
leaving a comment in the ticket. We'll be around to answer questions in the
channel, and to create new issues if there wasn't something that fit your
interests. When you submit a patch, we'll review it and give you some feedback.
Over time, we hope you'll learn a lot while supporting the project! Now get
hacking!

### Is this project ready for production?

Yes it is! It's been used in production for some time now. We think the level of
polish is excellent. We have enterprise support available too through:
[https://m9rx.com/](https://m9rx.com/).

Compared to some legacy existing automation tools out there, mgmt is a
relatively newer project. This shouldn't be seen as a detractor, we offer a
number of valuable, modern features which are not currently available elsewhere!

We're past our `1.0.0` release now. If major dangerous/scary/incompatible
changes happen, we'll bump that number. We'll bump the second number for bigger
changes that you might want to take note of. Nothing should break in a dangerous
way. Anything incompatible should be caught by the compiler safely or would be
prominently mentioned in the release notes. If we bump the last number it's for
smaller bug fixes, or other inconsequential features.

As with all software, bugs can occur, and while we make no guarantees of being
bug-free, there are a number of things we've done to reduce the chances of one
causing you trouble:

1. Our software is written in golang, which is a memory-safe language, and which
is known to reduce or eliminate entire classes of bugs.
2. We have a test suite which we run on every commit, and every 24 hours. If you
have a particular case that you'd like to test, you are welcome to add it in!
3. The mgmt language itself offers a number of safety features. You can
[read about them in the introductory blog post](https://purpleidea.com/blog/2018/02/05/mgmt-configuration-language/).

Having said all this, as with all software, there are still missing features
which some users might want in their production environments. We're working hard
to get all of those implemented, but we hope that you'll get involved and help
us finish off the ones that are most important to you. We are happy to mentor
new contributors too!

### Why did you use etcd? What about consul?

Etcd and consul are both written in golang, which made them the top two
contenders for my prototype. Ultimately a choice had to be made, and etcd was
chosen, but it was also somewhat arbitrary. If there is available interest,
good reasoning, *and* patches, then we would consider either switching or
supporting both, but this is not a high priority at this time.

### Can I use an existing etcd cluster instead of the automatic embedded servers?

Yes, it's possible to use an existing etcd cluster instead of the automatic,
elastic embedded etcd servers. To do so, simply point to the cluster with the
`--seeds` variable, the same way you would if you were seeding a new member to
an existing mgmt cluster.

The downside to this approach is that you won't benefit from the automatic
elastic nature of the embedded etcd servers, and that you're responsible if you
accidentally break your etcd cluster, or if you use an unsupported version.

### In `mgmt` you talk about events. What is this referring to?

Mgmt has two main concepts that involve "events":
1. Events in the [resource primitive](resource-guide.md).
2. Events in the [reactive language](language-guide.md).

Each resource primitive in mgmt can test (check) and set (apply) the desired
state that was requested of it. This is familiar to what is common with existing
tools such as `Puppet`, `Ansible`, `Chef`, `Terraform`, etc... In addition,
`mgmt` can also **watch** the state and detect changes. As a result, it never
has to waste time and cpu resources by polling to test and set state, leading to
a design which is algorithmically much faster than the existing generation of
tools.

To describe the set of resources to apply, mgmt describes this collection with a
language. In order to model the time component of infrastructure, we use a
special kind of language called an [FRP](https://en.wikipedia.org/wiki/Functional_reactive_programming).
This language has a built-in concept that we call "events", and which means that
we re-evaluate the relevant portions of the code whenever a value or function
has an event that tells us that it changed. The `R` in `FRP` stands for
reactive. This is similar to how a spreadsheet updates dependent cells when a
pre-requisite value is modified. [This article](https://en.wikipedia.org/wiki/Reactive_programming)
provides a bit more background.

Whenever any of the streams of values in the language change, the program is
partially re-evaluated. The output of any mgmt program is a [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph)
of resources, or more precisely, a stream of resource graphs. Since we have
events per-resource, we can efficiently switch from one desired-state resource
graph to the next without re-checking their individual states, since we've been
monitoring them all along.

One side-effect of all this, is that if a rogue systems administrator manually
changes the state of any managed resource, mgmt will detect this and attempt to
revert the change. This makes for excellent live demos, but is not the primary
design goal. It is a consequence of tracking state so that graph changes are
efficient. We implement the event detection via an intentional per-resource
[main loop](https://en.wikipedia.org/wiki/Event_loop) which can enable other
interesting functionality too!

Make sure to get rid of your rogue sysadmin! ;)

### Do I need to run `mgmt` as `root`?

No and yes. It depends. Nothing in mgmt explicitly requires root in the design,
however mgmt will require root only if the changes to your system that you want
it to make require root.

For example, if you use it to manage files that require root access to modify,
then you'll need root. If you only use it to manage files and resources
elsewhere, then it shouldn't need root. Many resources are perfectly usable
without root, and virtually all of my live demos are done without root.

### How can I run `mgmt` on-demand, or in `cron`, instead of continuously?

By default, `mgmt` will run continuously in an attempt to keep your machine in a
converged state, even as external forces change the current state, or as your
time-varying desired state changes over time. (You can write code in the mgmt
language which will let you describe a desired state which might change over
time.)

Some users might prefer to only run `mgmt` on-demand manually, or at a set
interval via a tool like `cron`. In order to do so, `mgmt` must have a way to
shut itself down after a single "run". This feature is possible with the
`--converger-timeout <N>` and `converged-exit` flags. You may specify these
flags, along with a number of seconds as the argument, and when there has been
no activity for that many seconds, the program will shutdown.

Alternatively, while it is not recommended, if you'd like to ensure the program
never runs for longer that a specific number of seconds, you can ask it to
shutdown after that time interval using the `--max-runtime` flag. This also
requires a number of seconds as an argument.

#### Example:

```
./mgmt run lang examples/lang/hello0.mcl --converger-timeout=5 --converged-exit
```

### Can I run `mgmt` for type-checking only?

Yes, you can use the `check` command with the mcl format check disabled, and it
will exit after type unification.

#### Example:

```
./mgmt check --tmp-prefix lang --skip-fmt examples/lang/hello0.mcl
```

It will also print how long it took on either success or failure. Keep in mind
that even if you pass type unification, an `mgmt` run can still fail later on
for other reasons, although these are mostly runtime considerations.

### Why is type unification happening twice with `mgmt run`?

When you use the `run` action, it runs all of the compile time checks (including
type unification) that are possible, and then packages everything up into a
deployable object and runs the same mechanism that `mgmt deploy` uses, sending
the deploy to itself. At this point, mgmt starts up as a server, and receives
the deploy. It will then need to run type unification again before running the
code.

You can skip the first type unification check by adding the `--skip-unify`
option to the lang frontend when using the `run` command.

You can also skip this check when running the `deploy` action, but if your code
doesn't pass, you might be deploying broken code. This is not recommended.

#### Example:

```
./mgmt run --tmp-prefix lang --skip-unify examples/lang/hello0.mcl
```

### Why does my file resource error with `no such file or directory`?

If you create a file resource and only specify the content like this:

```
file "/tmp/foo" {
	content => "hello world\n",
}
```

Then this will attempt to set the contents of that file to the desired string,
but *only* if that file already exists. If you'd like to ensure that it also
gets created in case it is not present, then you must also specify the state:

```
file "/tmp/foo" {
	state => $const.res.file.state.exists,
	content => "hello world\n",
}
```

Similar logic applies for situations when you only specify the `mode` parameter.

This all turns out to be more safe and "correct", in that it would error and
prevent masking an error for a situation when you expected a file to already be
at that location. It also turns out to simplify the internals significantly, and
remove an ambiguous scenario with the reversible file resource.

### Package resources error with: "The name is not activatable", what's wrong?

You may see an error like:

`main: error running auto edges: The name is not activatable`

This can happen because the mgmt `pkg` resource uses a library and daemon called
`PackageKit` to install packages. If it is not installed, then it cannot do its
work. On Fedora system you may wish to run `dnf install /usr/bin/pkcon` or on a
Debian system you may wish to run `apt install packagekit-tools`.

PackageKit is excellent because it provides both an API and an event system to
watch the package database for changes, and it abstracts away the differences
between the various package managers. If you'd prefer to not need to install
this tool, then you can contribute a native `pkg:rpm` and `pkg:deb` resource to
mgmt!

### When running mgmt, it says: "module path error: can't find a module path".

You might get an error along the lines of:

```
could not set scope: import scope `git://github.com/purpleidea/mgmt/modules/some_module_name/` failed: module path error: can't find a module path
```

This usually means that you haven't specified the directory that mgmt should use
when looking for modules. This could happen when using mgmt interactively or
when it's being run as a service. In such cases you may want the main invocation
to look something like:

```
mgmt run lang --module-path '/etc/mgmt/modules/' /etc/mgmt/main.mcl
```

### I get an error: "cannot open shared object file: No such file or directory".

Mgmt currently uses two libraries that depend on `.so` files being installed on
the host. Those are for `augeas` and `libvirt`. If those dependencies are not
present, then mgmt will not run. The complete error might look like:

```
mgmt: error while loading shared libraries: libvirt-lxc.so.0: cannot open shared object file: No such file or directory
```

or:

```
mgmt: error while loading shared libraries: libaugeas.so.0: cannot open shared object file: No such file or directory
```

or something similar. There are two solutions to this:

1. Use a build that doesn't include one or both of those features. You can build
that like: `GOTAGS="noaugeas novirt nodocker" make build`.

2. Install those dependencies. On a Fedora machine you might want to run:

```
dnf install libvirt-devel augeas-devel
```

On a Debian machine you might want to run:

```
apt install libvirt-dev libaugeas-dev
```

On an Arch machine you might want to run:

```
pacman -S libvirt augeas
```

### Why do function names inside of templates include underscores?

The golang template library which we use to implement the golang.template() func
doesn't support the dot notation, so we import all our normal functions, and
just replace dots with underscores. As an example, the standard `datetime.print`
function is shown within mcl scripts as datetime_print after being imported.

### On startup `mgmt` hangs after: `etcd: server: starting...`.

If you get an error message similar to:

```
etcd: server: starting...
etcd: server: start timeout of 1m0s reached
etcd: server: close timeout of 15s reached
```

But nothing happens afterwards, this can be due to a corrupt etcd storage
directory. Each etcd server embedded in mgmt must have a special directory where
it stores local state. It must not be shared by more than one individual member.
This dir is typically `/var/lib/mgmt/etcd/member/`. If you accidentally use it
(for example during testing) with a different cluster view, then you can corrupt
it. This can happen if you use it with more than one different hostname.

The solution is to avoid making this mistake, and if there is no important data
saved, you can remove the etcd member dir and start over.

### On running `make` to build a new version, it errors with: `Text file busy`.

If you get an error like:

```
cp: cannot create regular file 'mgmt': Text file busy
```

This can happen if you ran `make build` (or just `make`) when there was already
an instance of mgmt running, or if a related file locking issue occurred. To
solve this, shutdown and running mgmt process, run `rm mgmt` to remove the file,
and then get a new one by running `make` again.

### Type unification error with string interpolation.

Look carefully at the following code:

```
$num = 42
print "hello" {
	msg => "My favourite number is ${num}",
}
```

What's actually happening is that we can't unify `str + int`, because our
addition operator only supports `str + str`, `int + int`, and `float + float`.
The string interpolation feature can only combine strings. One solution is to
instead write:

```
$num = "42" # now this is a string
print "hello" {
	msg => "My favourite number is ${num}",
}
```

The first example will usually error with something along the lines of:

`unify error with: topLevel(func() { <built-in:concat> }): type error: int != str`

Now you know why this specific case doesn't work! We may reconsider allowing
other types to be pulled into interpolation in the future. If you have a good
case for this, then let us know.

### The run and deploy commands don't parse correctly when used with `--seeds`.

If you're running a command with `--seeds`, `--server-urls`, or `--client-urls`,
then make sure you are using an equals sign between the flag name and the value.
For example, if you were to run:

```
# wrong invocation!
mgmt deploy --no-git --seeds http://127.0.0.1:2379 lang code/test.mcl
```

Then the `--seeds` flag would interpret `lang` and `code/test.mcl` as additional
seeds. This flag as well as the other aforementioned ones all accept multiple
values. Use an equals sign to guarantee you enter the correct data, eg:

```
# better invocation! (note the equals sign)
mgmt deploy --no-git --seeds=http://127.0.0.1:2379 lang code/test.mcl
```

This is caused by a parsing peculiarity of the CLI library that we are using.
This is tracked upstream at: [https://github.com/alexflint/go-arg/issues/239](https://github.com/alexflint/go-arg/issues/239).
We have a workaround in place to mitigate it and attempt to show you a helpful
error message, but it's also documented here in the meantime. The error you will
see is: `cli parse error: missing equals sign for list element`.

### Can `mgmt` deploy a new version of mcl whenever that source changes?

The short answer is yes, but you probably don't want that. The reason is that
you can't reliably tell when a file update has *finished* changing. So if you
watch file system events, and react automatically to run a deploy, then you
might read a partial (truncated) version of the `mcl` code and deploy something
incorrect or dangerous.

If you do really want to "yolo" deploy, then you could first run `mgmt` locally:

```bash
mgmt run --tmp-prefix empty
```

and then run this dangerous inotify+deploy command to deploy to it:

```bash
while inotifywait -e close_write main.mcl; do mgmt deploy --no-git --seeds=http://127.0.0.1:2379 lang main.mcl; done
```

This could also deploy to a cluster of agents anywhere by changing the `--seeds`
endpoint, or as shown above, if you're just running a single node, this would
deploy to the standalone binary you've previously run.

### The docs speaks of `--remote` but the CLI errors out?

The `--remote` flag existed in an earlier version of mgmt. It was removed and
will be replaced with a more powerful version, which is a "remote" resource. The
code is mostly ready but it's not finished. If you'd like to help finish it or
sponsor the work, please let me know.

### Does this support Windows? OSX? GNU Hurd?

Mgmt probably works best on Linux, because that's what most developers use for
serious automation workloads. Support for non-Linux operating systems isn't a
high priority of mine, but we're happy to accept patches for missing features
or resources that you think would make sense on your favourite platform.

### Why aren't you using `glide`, `godep` or `go mod` for dependency management?

Vendoring dependencies means that as the git master branch of each dependency
marches on, you're left behind using an old version. As a result, bug fixes and
improvements are not automatically brought into the project. Instead, we run our
complete test suite against the entire project (with the latest dependencies)
[every 24 hours](https://docs.travis-ci.com/user/cron-jobs/) to ensure that it
all still works.

Occasionally a dependency breaks API and causes a failure. In those situations,
we're notified almost immediately, it's easy to see exactly which commit caused
the breakage, and we can either quickly notify the author (if it was a mistake)
or update our code if it was a sensible change. This also puts less burden on
authors to support old, legacy versions of their software unnecessarily.

Historically, we've had approximately one such breakage per year, which were all
detected and fixed within a few hours. The cost of these small, rare,
interruptions is much less expensive than having to periodically move every
dependency in the project to the latest versions. Some examples of this include:

* We caught the `go-bindata` swap before it was publicly known, and fixed it in:
[adbe9c7be178898de3645b0ed17ed2ca06646017](https://github.com/purpleidea/mgmt/commit/adbe9c7be178898de3645b0ed17ed2ca06646017).

* We caught the `codegangsta/cli` API change improvement, and fixed it in:
[ab73261fd4e98cf7ecb08066ad228a8f559ba16a](https://github.com/purpleidea/mgmt/commit/ab73261fd4e98cf7ecb08066ad228a8f559ba16a).

* We caught an un-announced libvirt API change, and promptly fixed it in:
[95cb94a03958a9d2ebf01df0821a8c13a4f3a28c](https://github.com/purpleidea/mgmt/commit/95cb94a03958a9d2ebf01df0821a8c13a4f3a28c).

If we choose responsible dependencies, then it usually means that those authors
are also responsible with their changes to API and to git master. If we ever
find that it's not the case, then we will either switch that dependency to a
more responsible version, or fork it if necessary.

Occasionally, we want to pin a dependency to a particular version. This can
happen if the project treats `git master` as an unstable branch, or because a
dependency needs a newer version of golang than the minimum that we require for
our project. In those cases it's sensible to assume the technical debt, and
vendor the dependency. The common tools such as `glide` and `godep` work by
requiring you install their software, and by either storing a yaml file with the
version of that dependency in your repository, and/or copying all of that code
into git and explicitly storing it. This project thinks that all of these
solutions are wasteful and unnecessary, particularly when an existing elegant
solution already exists: `[git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules)`.

The advantages of using `git submodules` are three-fold:
1. You already have the required tools installed.
2. You only store a pointer to the dependency, not additional files or code.
3. The git submodule tools let you easily switch dependency versions, see diff
output, and responsibly plan and test your versions bumps with ease.

Don't blindly use the tools that others tell you to. Learn what they do, think
for yourself, and become a power user today! That process led us to using
`git submodules`. Hopefully you'll come to the same conclusions that we did.

**UPDATE:**

After golang made it virtually impossible to build without `go.mod` stuff, we've
switched to it since golang 1.16. I still think the above approach was better,
and that the `go mod` tooling should have been a layer on top of git submodules
so that we don't grow yet another lock file format, and existing folks who are
comfortable with `git` can use those tools directly.

### Did you know that there is a band named `MGMT`?

I didn't realize this when naming the project, and it is accidental. After much
anguishing, I chose the name because it was short and I thought it was
appropriately descriptive. If you need a less ambiguous search term or phrase,
you can try using `mgmtconfig` or `mgmt config`.

It also doesn't stand for
[Methyl Guanine Methyl Transferase](https://en.wikipedia.org/wiki/O-6-methylguanine-DNA_methyltransferase)
which definitely existed before the band did.

### You didn't answer my question, or I have a question!

It's best to ask on [Matrix](https://matrix.to/#/#mgmtconfig:matrix.org) to see
if someone can help. If you don't get a response there, you can send a patch to
this documentation. I'll merge your question, and add a patch with the answer!
