Code Editors

Code Editors#

We’re flexible at the centre, but a majority of us use VSCode, or PyCharm.

Visual Studio Code#

VSCode is the most prominent IDE at the centre as it very quickly supports languages, extensions and code sharing.

VSCode - Python#

For Python, we recommend enabling the following settings:

  • Type Checking: basic (or strict, through your settings.json)

  • Ruff (by installing the ruff extension)

  • Auto Import Completions: true (makes life so much easier)

Note that ruff should automatically pick up the pyproject.toml in the root of the project you’re editing.

Alt text

Some cherry-picked settings.json options that might be handy:

{
  "python.analysis.typeCheckingMode": "strict",
  "[python]": {
    "editor.formatOnType": true,
    "editor.rulers": [
      {
        "column": 88,
        "color": "#ff9900"
      }
    ],
    "editor.formatOnSave": true,
    "editor.defaultFormatter": "charliermarsh.ruff",
    "editor.codeActionsOnSave": {
      "source.organizeImports": "explicit",
      "source.fixAll": "explicit"
    }
  }
}

Hail#

Hail in Visual Studio Code#

Python#

The Hail team allows line lengths up to 120 characters. In your Preferences, select the Workspace tab and search for Python › Formatting: Black Args. Set the arguments to --skip-string-normalization --line-length=120 to prevent black from adding lots of formatting changes that will make diffs hard to read.

After installing the Python extension, Visual Studio Code by default uses Pylance for code navigation. As Hail requires a non-trivial PYTHONPATH to resolve all modules, this is a little tricky to set up.

Add a .env file to the main Hail directory, which can be used to set up environment variables. Unfortunately, environment variable substitutions like $HOME don’t work in this file, so you’ll have to use absolute paths. Add the following line, adjusted accordingly for your system and user name:

PYTHONPATH="/Users/leo/hail:/Users/leo/hail/hail/python:/Users/leo/hail/gear:/Users/leo/hail/web_common"

At the time of writing, the Pylance build just got a fix to respect the $PYTHONPATH variable. However, it requires a new version of the Python (not Pylance!) extension. You can switch to that using the “Python: Insiders Channel” command.

Finally, you’ll have to set up a Python environment. Open a terminal in Visual Studio Code and run the following to install the required dependencies:

venv hail
pip3 install -r hail/python/requirements.txt
pip3 install -r docker/requirements.txt

# this puts the current python version on the clipboard
which python | pbcopy

Switch Visual Studio Code to this cvirtualonda environment using the “Python: Select Interpreter” command. You might also want to disable the inheritEnv setting.

If symbols still don’t get resolved properly, you might have to reload Visual Studio Code.

Scala#

If you’re working on Hail’s Scala code, the setup is slightly different. Assuming you’ve cloned the Hail repository to $HAIL, make sure that you open the hail subdirectory in Visual Studio Code, i.e. $HAIL/hail instead of $HAIL. That’s necessary because the Gradle configuration is located at $HAIL/hail/build.gradle.

  • Install the Scala (Metals) extension from the Marketplace.

  • Verify that build.gradle contains the Bloop-related changes from #111.

  • From the $HAIL/hail directory, run:

    make elasticsearchJar
./gradlew bloopInstall

  • Back in Visual Studio Code, run the Metals: Import build command. This might take a few minutes to complete.

  • Make sure everything worked out by running the Metals: Run doctor command. It should look like this: Metals doctor

Hail in PyCharm#

Create a new PyCharm project using the Hail source directory.

Then create a development codna environment:

venv hail-dev

pip3 install -r hail/python/requirements.txt
pip3 install -r docker/requirements.txt

# this puts the current python version on the clipboard
which python | pbcopy

Now you will need to configure PyCharm to use this environment. Go to the main PyCharm Preferences, which will open up a separate window. In the left pane, choose Project: hail>Project Interpreter. Now, in the main pane on the right, click the settings symbol (gear symbol) next to the field for Project Interpreter:

Choose Add in the menu that pops up, and select Existing environment:

Click on the ellipsis symbol next to the field for Interpreter. Enter the path to python interpreter (you could get this by running which python in the virtual environment), e.g. /Users/michael.franklin/.venv/analysis-runner/bin/python.

Now, same as for Visual Studio Code, you will need to set up PyCharm to resolve some Hail modules like gear and web_common that require a non-trivial PYTHONPATH. Go back to Preferences>Project: hail>Project Interpreter, and click the settings gear symbol again.

Now choose Show All in the menu that pops up. Pick the interpreter you are using for this project and click on the tree symbol at the bottom of the window (hovering over the symbol reveals it as “Show paths for the selected interpreter”):

Add the paths to the gear, web_common, batch folders in the repository root by clicking in the “plus” symbol:

Now PyCharm should resolve internal module imports in the code.

WDL#

There are two runners for WDL:

  1. Cromwell / WOMTool (for validating)

  2. MiniWDL

It’s recommended to validate with both WOMTool (the Cromwell validator) and MiniWDL, as sometimes they can return inconsistent results. You can validate a workflow called workflow.wdl with:

java -jar womtool-63.jar validate <workflow.wdl>
miniwdl check <workflow.wdl

WDL in Visual Studio Code#

In VSCode, we’d recommend installing the WDL DevTools extension for Visual Studio Code. It provides syntax highlighting, and uses MiniWDL to provide immediate feedback about syntax and linting.

Installing shellcheck will give you additional linting support within the command block.

Contents