refactor!: everything

Excuse the bold title, but here it is ### New "Command" language Overall the syntax is different, but is inspired and somewhat resembles the current one. The idea idea is to separate DSL from containers in parsing and in processing stages. The new mdsh `Command` consists of these parts: ``` [langname] <out_cmd> <in_cmd> [data_line] [data] ``` `in_cmd` defines how and where to source data, it can be one of three: - `<` — read file as is. The filepath is sourced from `data_line`, if `data` is available, it is read per line for filenames and each file is concatenated to previos one. - `$` — command execution. If the `data_line` is available, then it is executed as shell command. If the `data` is available it is passed to the command as via stdin (Closes https://github.com/zimbatm/mdsh/issues/57). If only `data` is available but not `data_line`, then the `data` is executed as shell script. - "empty command" aka "use data as is", concatenating `data_line` and `data`. In practice this is useful only for env variables setting `out_cmd` defines what to do with the data from `in_cmd`, it can be one of three: - `> lang` — produce code block with `lang` (similarly to current `as lang` statements). - `>` — produce raw markdown output fenced by comment-tags - `!` — expand data to shell variables with these 3 * 3 commands you get 9 combinations, for example: - `> < include.md` — read file and produce raw markdown - `> py < script.py` — read script.py and produce code block with language `py` - `> yml $ ./script.py foo $bar` — execute `script.py foo $bar` in shell and produce `yml` code block - `>$ ./gen-md.py` — execute `gen-md.py` and produce raw markdown - `! foo=$bar` — use `foo=$bar` as "raw data" and expand env variables that can be used in the next shell executions - `!< .env` — read `.env` and eval shell vars - `!$ ./gen-vars.py` — execute gen-vars and treat output as the list of shell variable assignments So it can do quite a lot of things and the underlying model is pretty simple, and even allows to do some useless things, like `> hello` — would produce an empty code block with `hello` language. ### Containers Commands can be put into containers, here's all of them: - inline code blocks: `langname` is skipped, parsing starts right from `out_cmd`, `data` is absent. Must start from new line and end with newline. - code blocks: ```` ```[langname] <out_cmd> <in_cmd> [data_line] [data] ``` ```` Source env vars: ````md ```env ! foo=$bar ``` ```` Execute script and produce yaml block (you can even put shebang at the top and use other than bash scripting languages. ````md ```sh > yaml $ echo 'foo: true' ``` ```` Run `data_line` as oneline command and pass code block to it via stdin, producing raw markdown. ````md ```> $ sed 's/.*/Hi, \0/' Bobby ``` ```` - oneline comments, similar to inline code blocks but hidden: `<!-- >< LICENSE.md -->` — includes LICENSE.md - multiline comment blocks, behaves similarly to code blocks, but `langname` is not needed ````md <!-- > yml $ echo 'hi: true' --> ```` - links. These are slightly deviate from the rest of containers: ``` [<out_cmd> <in_cmd> whatever here is ignored](<data_line>) ``` ### Key syntax design changes - Get rid of `as lang` constructions and move lang on the left so source code in code block can be properly highlighted - Keep old `>` `$` `!` with the same meaning but differently arranged, so a `Command` is a little pipeline of two stages now - Fix `>` behavior in code blocks, it was quite useless it produced raw markdown - Complement the set of supported containers with multiline comment blocks (Closes https://github.com/zimbatm/mdsh/issues/76) ### Key code changes - Rewrite everything to parses with `nom` library, now parsing code is easier to read and maintain and it's possible to do recursive parsing, f.x. in this PR comment tag fences can be in other comment tag fences, i.e. ```md <!-- BEGIN mdsh --> <!-- BEGIN mdsh --> hello <!-- END mdsh --> <!-- END mdsh --> ``` this would allow to include files that are processed by markdown and it shouldn't break - Horizontal layering: parsers in one pile, executors in another - Consistent error handling (with `anyhow`). It is currently a mixture of `io::Error`s, `unrwap`s, `panic`s, `expect` and custom `die` function. - Removed regexes, they are too hard to read and maintain compared to parser combinators - Extensive use of `Write` and `Read` that could potentially give *some* performance boost in *some* cases - Fencing with comment tags everything produced by mdsh. It makes parsing much simpler, also closes https://github.com/zimbatm/mdsh/issues/29 - Env vars substitution changed from bash expansion to [`shellexpand`](https://docs.rs/shellexpand/latest/shellexpand/). It supports simple like `${foo-def}` expressions, but not arbitrary code execution like `$(date)`. I think for this real scripts (`$` command) should be used if it is needed - Added generation of test cases that also acts as documentation (see `spec.{clear,processed}.md`)