CNN scripts

[dfilan]

Description

Add config options to train CNNs on image environments. This also involves letting configs set environment wrappers, so that environments like Atari can be appropriately wrapped.

Testing

Ran some scripts of interest locally, ran pytest tests/scripts/test_scripts.py.

TODOs:

• Add nice configs for all algorithms, not just preference comparisons.
• Add tests for CNN+image environment scripts.

[codecov]

Codecov Report

Merging #563 (d90d734) into master (1dd4c8f) will decrease coverage by 0.35%.
The diff coverage is 95.91%.

❗ Current head d90d734 differs from pull request most recent head 8f1d6e5. Consider uploading reports for the commit 8f1d6e5 to get more accurate results

@@            Coverage Diff             @@
##           master     #563      +/-   ##
==========================================
- Coverage   97.51%   97.15%   -0.36%     
==========================================
  Files          85       85              
  Lines        8316     7689     -627     
==========================================
- Hits         8109     7470     -639     
- Misses        207      219      +12     

Impacted Files	Coverage Δ
...ion/scripts/config/train_preference_comparisons.py	85.71% <88.88%> (+0.38%)	⬆️
src/imitation/scripts/config/train_rl.py	80.00% <88.88%> (+1.05%)	⬆️
src/imitation/policies/base.py	100.00% <100.00%> (ø)
src/imitation/scripts/common/common.py	97.29% <100.00%> (-0.24%)	⬇️
src/imitation/scripts/common/reward.py	98.70% <100.00%> (+0.05%)	⬆️
src/imitation/scripts/common/train.py	100.00% <100.00%> (ø)
src/imitation/scripts/train_rl.py	100.00% <100.00%> (ø)
tests/scripts/test_scripts.py	100.00% <100.00%> (ø)
src/imitation/algorithms/density.py	86.11% <0.00%> (-8.38%)	⬇️
src/imitation/scripts/parallel.py	59.64% <0.00%> (-6.39%)	⬇️
... and 71 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[dfilan]

It's proven tricky for me to get tests working for DAgger, AIRL, and GAIL, and it's low-priority for the project that I'm currently working on, so I'll stop working on this PR for the time being.

[dfilan]

One minor TODO: use the seals versions of the Atari environments.