Friday, April 7, 2017

How to generate documentation with Sphinx

Sphinx is a documentation generator that can help building great docs for our projects in a very short time. In this tutorial I will show how to use it to build docs starting from pre-existing Python docstrings.

Installing and configuring Sphinx

NOTE: If you are developing your Python package using virtualenv, I recommend installing Sphinx directly inside your virtual environment (see explanation at the end of the article).

In order to start working with Sphinx, we need to install it first. Unsurprisingly, this is as easy as:

pip install sphinx

We can now proceed and generate all the files that will be used by Sphinx to build our documentation. From terminal:

sphinx-quickstart

At this point the script will ask us to specify more details about our configuration. Here are some of the basic customizations we will be asked about:

  • Root path for the documentation: I usually choose docs/, to keep documentation separate from the rest of the project;
  • Separate source and build directories: choosing y will create two separate folders (build/ and source/) for our docs. I think this is the best choice if we want to keep everything in order;
  • Project name;
  • Author name(s);
  • Project version: I highly suggest to take a look at Semantic Versioning to choose the most suitable version at this stage;
  • Create Makefile?: choosing y will make things a bit easier later.

Sphinx ships with some extremely useful extensions that will make your life much easier. It is therefore important to choose the right ones for your needs. I usually use autodoc, todo, viewcode and githubpages. Note that these preferences can also be added to the conf.py file later.

In order to be quicker, we could also specify all of the above preferences in one command as follows:

sphinx-quickstart --sep -p 'Project Name' -a 'Author Name' -v '0.1.0' --makefile --ext-autodoc --ext-todo --ext-viewcode --ext-githubpages docs/

You should now see a structure like the following for your docs folder:

docs
├── Makefile
├── build
└── source
    ├── _static
    ├── _templates
    ├── conf.py
    └── index.rst

Building the docs

We can now proceed and let Sphinx build all the HTML files for our docs.

NOTE: HTML is not the only available format. You can use LaTeX, text, XML, and many others (see sphinx-build buildername options).

In order to do so, we will use the make command that we asked for during the configuration step above. Place yourself within the docs/ folder and issue the following command:

make html

NOTE: if you do not want to change directory, you can also use make -C docs/ html, where -C specifies the directory inside of which you want to issue the command.

At this point your build should have succeeded and your HTML files should be in docs/build/html. You can therefore open index.html in your browser and start taking a look around at your brand new docs. It won’t be long until you realize that something is missing!


Generating documentation from docstrings

What we did so far was to simply generate a sort of scaffolding for our documentation. However, we never explicitly told Sphinx to use our docstrings to automatically populate our docs! This is exactly what the sphinx-apidoc command does. From terminal:

sphinx-apidoc <path-to-project-package> -o docs/source -f -M

The above command uses the following options (you can omit them if you want):

  • -f, --force: overwrite all files generated by Sphinx;
  • -M: put module documentation before submodule documentation.

Customizing conf.py

Several customizations can be done to our docs to make them look shiny. The main place where magic happens is the conf.py file within our documentation folder. I will now share a few tricks that could help improving your documentation.


Hiding full module names


The vanilla configuration will produce an output where classes and methods described by the docs are nested inside their module name. However, all these entities will be prepended with their full namespace. For example:

module.long.full.name module

Your module description

class module.long.full.name.MyClass

As you can see, the full module name (module.long.full.name) is prepended to both the module (above) and the class (below). Since the class belongs to the module, we could reasonably prefer to hide the full module name and just keep the class name (MyClass) instead. In order to do so, we can add the following line to our conf.py configuration file:

# Do not prepend names to object names
add_module_names = False

We subsequently have to re-run the sphinx-apidoc and make commands as above to rebuild the HTML output files.


Changing Sphinx docs theme


Sphinx ships with several builtin themes that we can choose from. Additionally, we can also choose a customized theme (or create our own). In my case, I often go for the Sphinx ReadTheDocs theme. If you want to use it as well, install the sphinx_rtd_theme package:

pip install sphinx_rtd_theme

then add the following lines to your conf.py:

# Use ReadTheDocs theme
import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

We can now run again the sphinx-apidoc and make commands as seen above.


More


Many other customizations can be used to improve our documentation. Please make sure to read the Sphinx build configuration file docs if you want to achieve the best results for your project.


Notes

Why installing Sphinx inside virtualenv?


The make command that we use calls sphinx-build, which in turn uses the default Python interpreter. This means that if we install Sphinx inside the virtual environment, sphinx-build will correctly call the Python version in use for our project. Why is it important? The project needs to be installed in order to be found by Sphinx; since we probably installed it using pip install -e, Sphinx will only be able to find it with regards to the Python version used by our current virtualenv.


Sources

Thursday, October 20, 2016

How To Override Sublime Text Packages Shortcuts and Preferences

I recently began using a Sublime Text 3 package that automatically generates inline YARD documentation for my Ruby code (Yardgen). The only problem with this package is that the key bindings it provides are overriding other Sublime Text shortcuts that I want to keep.

Since this plugin is packed in a zip file, it is not possible to simply edit one of its keymap files (see Package Control - Customizing Packages). At the same time, unpacking the original zipped file and creating a new zip file would not work.

Following are the steps that you need in order to solve this problem and, more generally, to override Sublime Text 3 packed packages preferences.

NOTE: If your package is overriding some default binding you wanted to keep, but you do not know the command it was binded to, please refer to my previous post.

The steps!

Install your package (Yardgen, in my case) using the Package Manager or any other method you prefer.

Your zipped package file should be now placed within the folder ~/.config/sublime-text-3/Installed Packages/<your-package>.sublime-package.

Check the content of the package by unzipping it (just make sure to keep the original zipped file).

In my case I want to edit one of the default Yardgen keybindings. Key bindings for Linux are usually stored in the Default (Linux).sublime-keymap file within the Yardgen package archive. This is the file content:

[
  { "keys": ["ctrl+enter"], "command": "yard_gen"},
]

We now have to create a folder named Yardgen inside ~/.config/sublime-text-3/Packages. We can then place inside it our new key bindings file that will override the default package behaviour:

cd ~/.config/sublime-text-3/Packages
mkdir Yardgen
gedit "Default (Linux).sublime-keymap"

Now copy the following json content in Default (Linux).sublime-keymap:

[
  { "keys": ["ctrl+alt+shift+enter"], "command": "yard_gen"},
]

As you can see, we just associated the yard_gen command to a different key sequence. You can obviously choose the one that best suits your needs.

At the moment, the yard_gen command is binded to two different key sequences: the default one, still present in the original package, and the new one we just defined. In my case, I’m not ok with this. Yardgen key bindings have in fact overridden the default CTRL + Enter behaviour, and I want to have it back.

If you too want to restore the default binding, you just need to figure out which command it was originally associated with and then explicitly add its binding to the Key Bindings preference file. For more details, please refer to my other post: How To Find out Sublime Text Key Binding Commands.


References

Tuesday, October 18, 2016

How To Find out Sublime Text Key Binding Commands

I just happened to download an awesome package for Sublime Text. Unluckily, this package uses a keyboard shortcut that overwrites a default binding I really need (the CTRL + Enter command that adds a line below the current line). I therefore want to restore the normal ST behaviour and, in order to do that, I need to figure out which command was originally called when using that shortcut.

If a new package is overwriting some default binding you want to keep, but you do not know the command it was binded to, please keep reading.

NOTE: In order to make these steps work, you need to temporarily uninstall the package that is currently overwriting your key binding.


The Steps


Open Sublime Text console. To do so, you can either click on View > Show Console or use the CTRL + ` shortcut.

We can now activate the console log to show every command we run. Give the following command in the Sublime Text console:

sublime.log_commands(True)

Now press (in the appropriate context) the key sequence that you want to analyze. For example, if you want to find out the command associated with CTRL + Enter, press that sequence of keys while editing a file. In this case you will see something like the following appear in your console:

command: run_macro_file {"file": "res://Packages/Default/Add Line.sublime-macro"}

That is the command we are looking for. We can now deactivate the command logging feature before we proceed:

sublime.log_commands(False)

If you need to overwrite some package key bindings and restore the old ones, you can simply bind your old key sequence to that command. In the CTRL + Enter case, add this entry to your Default (Linux).sublime-keymap json file (Preferences > Key Bindings):

{ "keys": ["ctrl+enter"],
  "command": "run_macro_file",
  "args": {
    "file": "res://Packages/Default/Add Line.sublime-macro"
  }
}

If you also want to keep your new package shortcuts, binding them to a different key binding, please refer to How To Override Sublime Text Packages Shortcuts and Preferences.


References

Thursday, February 18, 2016

How To Make Terminator Behave Like Guake (Ubuntu)

Terminator is a tool to arrange multiple terminals in a single window, structured on a customizable grid.

When I started using Terminator on Ubuntu, I missed two key features that I had with Guake: an hide/show shortcut, and the possibility to hide its icon from the alt-tab list of running applications.

Hide/Show Terminator like Guake

This solution has been proposed on StackOverflow (see references). I will just copy the code with its attributions and some minor changes.

  • Install xdotool and wmctrl:

    sudo apt-get install xdotool wmctrl
  • Create a file /usr/bin/launch_focus_min.sh.

  • Add the following content to the file:

    #!/bin/bash
    #
    # This script does this:
    # launch an app if it isn't launched yet,
    # focus the app if it is launched but not focused,
    # minimize the app if it is focused.
    #
    # by desgua - 2012/04/29
    # modified by olds22 - 2012/09/16
    #  - customized to accept a parameter
    #  - made special exception to get it working with terminator
    
    # First let's check if the needed tools are installed:
    tool1=$(which xdotool)
    tool2=$(which wmctrl)
    
    if [ -z $tool1 ]; then
      echo "Xdotool is needed, do you want to install it now? [Y/n]"
      read a
      if [[ $a == "Y" || $a == "y" || $a = "" ]]; then
        sudo apt-get install xdotool
      else
        echo "Exiting then..."
        exit 1
      fi
    fi
    
    if [ -z $tool2 ]; then
      echo "Wmctrl is needed, do you want to install it now? [Y/n]"
      read a
      if [[ $a == "Y" || $a == "y" || $a = "" ]]; then
        sudo apt-get install wmctrl
      else
        echo "Exiting then..."
        exit 1
      fi
    fi
    
    
    # check if we're trying to use an app that needs a special process name
    # (because it runs multiple processes and/or under a different name)
    app=$1
    if [[ $app == terminator ]]; then
      process_name=usr/bin/terminator
    else
      process_name=$app
    fi
    
    # Check if the app is running (in this case $process_name)
    
    #pid=$(pidof $process_name) # pidof didn't work for terminator
    pid=$(pgrep -f $process_name)
    
    # If it isn't launched, then launch
    
    if [ -z $pid ]; then
      $app
    
    else
    
      # If it is launched then check if it is focused
    
      foc=$(xdotool getactivewindow getwindowpid)
    
      if [[ $pid == $foc ]]; then
    
        # if it is focused, then minimize
        xdotool getactivewindow windowminimize
      else
        # if it isn't focused then get focus
        wmctrl -x -R $app
      fi
    fi
    
    exit 0
    
  • Make the script executable:

    sudo chmod +x /usr/bin/launch_focus_min.sh
  • Assign the script to a keyboard shortcut. You can do it on Ubuntu using the Keyboard settings:

    Keyboard > Shortcuts > Customized Shortcuts

    From there, you can now add the following custom command:

    /usr/bin/launch_focus_min.sh terminator

    Note that the **launch_focus_min.sh** script can be also used with applications other than **terminator**.

Hide Terminator icon from Alt-Tab menu

  • Download CompizConfig Settings Manager and the compiz-plugins-extra package

    sudo apt-get install compizconfig-settings-manager compiz-plugins-extra
  • Open CompizConfig Settings Manager.

  • Click on Manage Windows.

  • Tick the box next to Window Rules to enable them.

  • Click on Window Rules.

  • Fill in the Skip Taskbar and Skip Pager fields with the following:

    (name=terminator) & class=Terminator

    Note - Here is the meaning of the two fields we filled:

    Skip taskbar These windows will not show up on the task bar (the list of buttons you click on to switch between open windows).
    Skip pager These windows will not show up on the desktop pager (the applet you click on to switch to desktops/workspaces/viewports).

Terminator should now disappear from the Alt-Tab menu.


References

Tuesday, July 14, 2015

Sentiment Analysis lexicons and datasets

Last update: Monday, October 19, 2015.

This is a list of some available lexicons and corpora for Sentiment Analysis (also called Opinion Mining). Il will try to keep this list updated as much as possible. Have fun!

Lexicons

Datasets


Lexicons


Datasets