Option to Install only specific programs

[aswinkumar1999]

Thought the setup scripts are made for quick setups, I would prefer the option of choosing or ignoring certain programs, Especially Anaconda in my case.

Would be cool if we can have a menu where the default is to install everything and we get an option to not install certain programs.

[rsnk96]

Wondering what might be the best way to implement this, at least for the major libraries like anaconda, etc

@aswinkumar1999 @rajat2004 what's your take, command line arguments, or a menu like this or using ncurses, or something else?

[rajat2004]

IMO, env variables might be the better way to go about this, similar to how CIINSTALL is used currently.

Having a dialog requires user input, which could cause problems when running non-interactively such as Docker, plus just executing the script and then just going somewhere while it runs is nice :)
Dealing with command-line arguments to the script such as --no-install-conda will be a pain, unless we're using something like argparse in Python, which I don't think there's any easy way to do in shell scripts. Changing the scripts to Python also doesn't seem to be very useful

Something like INSTALL_CONDA=no ./1-BasicSetup.sh, could add usage details which would list the variables and the default behaviour for --help option

[aswinkumar1999]

I feel like it would be better with a menu like a dialog box, something like a Python wrapper for all the scripts.
Where I can have both command-line arguments and menu and issue something like a --unattended for docker, Travis or for unattended builds.

Changing the scripts to Python also doesn't seem to be very useful

I am not sure if i understand the above properly because i feel like adding Python to it could possible allow us to some parts of Install Dynamic, such as compiling OpenCV for the specific GPU Arch for CUDA and Fetching the Latest CUDA and other conditional steps which would be hard with bash.

Thought the possibilities are a lot, I guess initially it will be slightly painful the make the complete switch.

[rajat2004]

Problem with command-line arguments in a shell script I think is that it's very painful to deal with. One or 2 options such as --help or --verbose are okay, but after that it would become extremely complicated and difficult to handle.
This would essentially get compounded since options can be reordered, short-form as well, etc...
Env variables would be descriptive, no problem with reordering and handling is quite simple

Python would be simpler here, with libraries like argparse plus others are there for GUI as well

But apart from this, right now I'm not seeing many more benefits of going with Python. Most of the things being done are shell commands, which in Python would require using os.system() or subprocess for each one, plus careful handling when we do want to affect the environment of the current shell, etc.

Even getting details like specific GPU arch, etc. will mostly require using shell commands, and piping + options like grep, etc. will be easier to deal with in bash script than Python

This is just my current thinking, would love to have your ideas on how to go about all this, places where I'm wrong, other options, etc.

[aswinkumar1999]

So Me and @rajat2004 had a talk, and I realised how indescriptive my previous comment, I'm adding this as a followup comment with an example.

So , When I meant

Python wrapper for all the scripts.

I meant it more like a top-level one where the scripts stay the same and python is going to act like a parser deciding which lines of the bash script is it is gonna run.

Ex: I'm taking the Basics scripts.

$ python3 setup_ubuntu.py
          -> [x] Basics 
                      [x] ZSH 
                              -> Some Parameter = Derfault Value 
                              # Can Change it if desired  
                      [x] Git
                      [x] Tilda
                      [ ] Anaconda 
                      .
                      .       

And this 1-BasicSetUp.sh is gonna have something like

.
.
.
####### ZSH ( parameters) ########
~ All commands needed to install ZSH go Here ~
###### END ########
.
####### Git ########
execute sudo apt-get install git -y
###### END ########
.
.

To put it simply, this python code will then look into the 1-BasicSetUp.sh and parse the list and run all the lines of the bash script which are selected.

Kind of like a Function call.

And say adding an argument like :

$ python3 setup_ubuntu.py --all 

# or 

$ python3 setup_ubuntu.py --unattended

Will Disable Interactive mode completely and will get the entire script running.

And for the Feasibility check, most part of it should be fine but I am not exactly sure about handling Environment Variables here.

And

This is just my current thinking, would love to have your ideas on how to go about all this, places where I'm wrong, other options, etc.

+1

[MukundVarmaT]

@aswinkumar1999 This would be a wonderful feature. I am not sure if this is done or not! But I did come across a python package called inquirer which basically gives a nice list of options with the option of multi-selecting them (using the right arrow).

Like for example:

questions = [
  inquirer.Checkbox('Code editor Installation',
                    message="Do you want to install?",
                    choices=['VS Code', 'Sublime', 'Atom', 'Pycharm'],
                    ),
]

answers = inquirer.prompt(questions)

This is what it would look like --->

[rsnk96]

This does sound like a promising lead now. Only part where I'm still unsure in is if there's some drawback in converting the whole script to a Python script that we're missing

Any opinions @rajat2004 @aswinkumar1999 @MukundVarmaT?

[MukundVarmaT]

I am not sure but how do we ensure that the user has packages required to run the python setup scripts. Like in the above case, how will a user have the package inquirer already installed on their system?

There is something called pyinstaller and it creates a self-contained executable file with all the required libraries etc etc. So the person running it need not even have python on their system to run the executable. Not sure why I brought this up but I guess we need to look at something similar to that.

[aswinkumar1999]

Yeah, pyinstaller seems to be a good idea but I doubt it for our case, especially considering the setup scripts are for Ubuntu, Python3 has been a part of it since Ubuntu 14.04.

I came across two ideas.

• If most of our support is for Ubuntu >= 14.04, then we can simply call a subprocess at the start of python script to install the packages that we'll use.
• If Ubuntu version shouldn't matter or it should be compatible with other similar distributions that these scripts can run and if we want a more formal way to do it, then we can have a base bash script which checks for python3 installation and installs other libraries and then runs the python code and cleans up everything in the end.

[rsnk96]

Option 2 might add some otherwise avoidable overhead.

As suggested by everyone on this thread, we could make Python an interactive parser for choosing which components of the script get installed, and use inquirer or any equivalent package(perhaps installed by Option 1) to do the same. Would like if you guys could contribute this 😄 , I'll probably start off by refactoring the existing bash scripts so that they become easier to parse in Python if you guys are up for it!

[MukundVarmaT]

Sure count me in. Let me know how I can help.
Just one additional thing, maybe we can try to use python for displaying the terminal outputs. Cause as far as I remember the execute function had some issues, I remember one thing like let's say a package doesn't have release files it just exits out of the script without showing anything. Some changes there would help as well.

[aswinkumar1999]

Sure count me in. Let me know how I can help.

+1

Sure , Let's do this.... I do have certain questions before we proceed.

• I do remember @rajat2004 telling me how much time he worked on sorting out the environment variables issue and stuff related to that , so I feel that is something we need to take care of... To put my questions properly.

  • Can I run all five scripts together in the same shell without encountering any variable problems?
  • Can I run parts of the script separately in a shell and not worry about it ?
    • Ex : Does Installation of XYZ depends on some other stuff which is shared for a 2 or more installations ? If yes ? Do we simply have two copies of it ? If Yes ? Then is there any chance it can affect if I take the scripts and run them individually like how I do now ? Because I want us to write a parser and not change these scripts entirely and retain the functionality that we have now too just in case someone prefers it that way ...

• Are there any major dependencies to a take care of ? Like FFMPEG can be used in OpenCV , but I might not be a fan of installing FFMPEG from source rather would just prefer OpenCV from source or Vice-versa ... How do I manage those ?

• How does the Python Script work ?

  • Install every software using a new subprocess ?
  • Put all the selected ones into a bash script and run the script ( Useful in case of shared Env variables ) ?

[varun19299]

Some thoughts (if this is still being actively pursued):

• How about an accompanying config file that would let people toggle needed flags. This could be a yaml file for instance.

• Each config entry could be followed with corresponding command line flag (same name). You can follow the following heirarchy:

Command line > YAML > Defaults (use a function to successively, similar to what sacred or hydra does, although very different objectives).

• I guess Go has become popular for such setup scripts (atleast rclone and a few other projects seem to use it). Python might have an overhead, but could be an insignificant one.

• Another issue with python: If I'm using these scripts on 16.04, I'm stuck with python 2.7. Which means, the code base cannot use 3.8, unless you explictly ask the user to pre-install.

Again, might be not-so-important.

[rsnk96]

I guess Go has become popular for such setup scripts (atleast rclone and a few other projects seem to use it). Python might have an overhead, but could be an insignificant one

Is it possible to cross-compile Go programs for different platforms? Also, we need people proficient in Go 😅

How about an accompanying config file that would let people toggle needed flags. This could be a yaml file for instance.

The major hurdle in both this method as well as writing a configuration-setter in Python is to build a parser / reuse some parser that's already out there. Haven't gotten around to looking that up yet.

Another issue with python: If I'm using these scripts on 16.04, I'm stuck with python 2.7. Which means, the code base cannot use 3.8, unless you explictly ask the user to pre-install.

Good point, this would be a deal breaker. If something's being written in Python, this'll have to be taken into consideration. I'm also wondering what is to be done in OS images / docker images where Python doesn't come out of the box (In the Alpine images for instance). But I guess we can't cater to all use cases