galaxy.tools.deps package

Submodules

galaxy.tools.deps.brew_exts module

class galaxy.tools.deps.brew_exts.BrewContext(args=None)[source]

Bases: object

exception galaxy.tools.deps.brew_exts.CommandLineException(command, stdout, stderr)[source]

Bases: exceptions.Exception

class galaxy.tools.deps.brew_exts.EnvAction(keg_root, action_description)[source]

Bases: object

static build_env(env_actions)[source]
modify_environ(environ)[source]
to_statements()[source]
class galaxy.tools.deps.brew_exts.RecipeContext(recipe, version, brew_context=None)[source]

Bases: object

cellar_path
static from_args(args, brew_context=None)[source]
tap_path
galaxy.tools.deps.brew_exts.brew_deps(package)[source]
galaxy.tools.deps.brew_exts.brew_execute(args, env=None)[source]
galaxy.tools.deps.brew_exts.brew_head_at_commit(*args, **kwds)[source]
galaxy.tools.deps.brew_exts.brew_head_at_version(*args, **kwds)[source]
galaxy.tools.deps.brew_exts.brew_info(recipe)[source]
galaxy.tools.deps.brew_exts.brew_versions_info(package, tap_path)[source]
galaxy.tools.deps.brew_exts.build_env_actions(deps, cellar_root, cellar_path, relaxed=None, custom_only=False)[source]
galaxy.tools.deps.brew_exts.build_env_statements(cellar_root, cellar_path, relaxed=None, custom_only=False)[source]
galaxy.tools.deps.brew_exts.build_env_statements_from_recipe_context(recipe_context, **kwds)[source]
galaxy.tools.deps.brew_exts.commit_for_version(recipe_context, package, version)[source]
galaxy.tools.deps.brew_exts.ensure_brew_on_path(args)[source]
galaxy.tools.deps.brew_exts.execute(cmds, env=None)[source]
galaxy.tools.deps.brew_exts.extended_brew_info(recipe)[source]
galaxy.tools.deps.brew_exts.git_execute(args)[source]
galaxy.tools.deps.brew_exts.load_versioned_deps(cellar_path, relaxed=None)[source]
galaxy.tools.deps.brew_exts.main()[source]
galaxy.tools.deps.brew_exts.print_versioned_deps(recipe_context, recipe, version)[source]
galaxy.tools.deps.brew_exts.recipe_cellar_path(cellar_path, recipe, version)[source]
galaxy.tools.deps.brew_exts.unversioned_install(package)[source]
galaxy.tools.deps.brew_exts.versioned_install(recipe_context, package=None, version=None, installed_deps=[])[source]
galaxy.tools.deps.brew_exts.which(file)[source]

galaxy.tools.deps.brew_util module

brew_exts defines generic extensions to Homebrew this file builds on those abstraction and provides Galaxy specific functionality not useful to the brew external commands.

class galaxy.tools.deps.brew_util.HomebrewRecipe(recipe, version, tap)[source]

Bases: object

galaxy.tools.deps.brew_util.requirement_to_recipe(requirement)[source]
galaxy.tools.deps.brew_util.requirements_to_recipe_contexts(requirements, brew_context)[source]
galaxy.tools.deps.brew_util.requirements_to_recipes(requirements)[source]

galaxy.tools.deps.commands module

Generic I/O and shell processing code used by Galaxy tool dependencies.

galaxy.tools.deps.commands.argv_to_str(command_argv, quote=True)[source]

Convert an argv command list to a string for shell subprocess.

If None appears in the command list it is simply excluded.

Arguments are quoted with shlex_quote. That said, this method is not meant to be used in security critical paths of code and should not be used to sanitize code.

exception galaxy.tools.deps.commands.CommandLineException(command, stdout, stderr, returncode)[source]

Bases: exceptions.Exception

An exception indicating a non-zero command-line exit.

galaxy.tools.deps.commands.download_command(url, to='-', quote_url=False)[source]

Build a command line to download a URL.

By default the URL will be downloaded to standard output but a specific file can be specified with the to argument.

galaxy.tools.deps.commands.execute(cmds)[source]

Execute commands and throw an exception on a non-zero exit.

Return the standard output if the commands are successful

galaxy.tools.deps.commands.redirect_aware_commmunicate(p, sys=<module 'sys' (built-in)>)[source]

Variant of process.communicate that works with in process I/O redirection.

galaxy.tools.deps.commands.redirecting_io(sys=<module 'sys' (built-in)>)[source]

Predicate to determine if we are redicting stdout in process.

galaxy.tools.deps.commands.shell(cmds, env=None, **kwds)[source]

Run shell commands with shell_process and wait.

galaxy.tools.deps.commands.shell_process(cmds, env=None, **kwds)[source]

A high-level method wrapping subprocess.Popen.

Handles details such as environment extension and in process I/O redirection.

galaxy.tools.deps.commands.which(file)[source]

galaxy.tools.deps.conda_compat module

Compat. layer with conda_build/verify if Galaxy/galaxy-lib not installed through conda.

In general there are utilities available for Conda building and parsing that are high-quality and should be utilized when available but that are only available in conda channels and not in PyPI. This module serves as a PyPI capable interface to these utilities.

galaxy.tools.deps.conda_compat.MetaData

alias of galaxy.tools.deps.conda_compat._MetaData

galaxy.tools.deps.conda_compat.raw_metadata(recipe_dir)[source]

Evaluate Conda template if needed and return raw metadata for supplied recipe directory.

galaxy.tools.deps.conda_util module

class galaxy.tools.deps.conda_util.CondaContext(conda_prefix=None, conda_exec=None, shell_exec=None, debug=False, ensure_channels='', condarc_override=None, use_path_exec=False, copy_dependencies=False, use_local=False)[source]

Bases: galaxy.tools.deps.installable.InstallableContext

activate
can_install()[source]

Check preconditions for installation.

can_install_conda()[source]

If conda_exec is set to a path outside of conda_prefix, there is no use installing conda into conda_prefix, since it can’t be used by galaxy. If conda_exec equals conda_prefix/bin/conda, we can install conda if either conda_prefix does not exist or is empty.

conda_build_available
conda_info()[source]
conda_version
deactivate
ensure_conda_build_installed_if_needed()[source]
env_path(env_name)[source]
envs_path
exec_clean(args=[], quiet=False)[source]

Clean up after conda installation.

Return the process exit code (i.e. 0 in case of success).

exec_command(operation, args, stdout_path=None)[source]

Execute the requested command.

Return the process exit code (i.e. 0 in case of success).

exec_create(args, allow_local=True, stdout_path=None)[source]

Return the process exit code (i.e. 0 in case of success).

exec_install(args, allow_local=True, stdout_path=None)[source]

Return the process exit code (i.e. 0 in case of success).

exec_remove(args)[source]

Remove a conda environment using conda env remove -y –name args.

Return the process exit code (i.e. 0 in case of success).

export_list(name, path)[source]

Return the process exit code (i.e. 0 in case of success).

has_env(env_name)[source]
installable_description = 'Conda'
is_conda_installed()[source]

Check if conda_exec exists

is_installed()[source]

Return bool indicating if the configured software is installed.

parent_path
class galaxy.tools.deps.conda_util.CondaTarget(package, version=None, channel=None)[source]

Bases: object

install_environment

The dependency resolution and installation frameworks will expect each target to be installed it its own environment with a fixed and predictable name given package and version.

package_specifier

Return a package specifier as consumed by conda install/create.

galaxy.tools.deps.conda_util.install_conda(conda_context, force_conda_build=False)[source]
galaxy.tools.deps.conda_util.install_conda_target(conda_target, conda_context, skip_environment=False)[source]

Install specified target into a its own environment.

Return the process exit code (i.e. 0 in case of success).

galaxy.tools.deps.conda_util.requirements_to_conda_targets(requirements)[source]

galaxy.tools.deps.containers module

class galaxy.tools.deps.containers.AppInfo(galaxy_root_dir=None, default_file_path=None, outputs_to_working_directory=False, container_image_cache_path=None, library_import_dir=None, enable_beta_mulled_containers=False, containers_resolvers_config_file=None, involucro_path=None, involucro_auto_init=True)[source]

Bases: object

class galaxy.tools.deps.containers.Container(container_id, app_info, tool_info, destination_info, job_info, container_description)[source]

Bases: object

containerize_command(command)[source]

Use destination supplied container configuration parameters, container_id, and command to build a new command that runs input command in container.

resolve_dependencies
shell
class galaxy.tools.deps.containers.ContainerFinder(app_info)[source]

Bases: object

find_best_container_description(enabled_container_types, tool_info)[source]

Regardless of destination properties - find best container for tool.

Given container types and container.ToolInfo description of the tool.

find_container(tool_info, destination_info, job_info)[source]
class galaxy.tools.deps.containers.ContainerRegistry(app_info)[source]

Bases: object

Loop through enabled ContainerResolver plugins and find first match.

find_best_container_description(enabled_container_types, tool_info)[source]

Yield best container description of supplied types matching tool info.

class galaxy.tools.deps.containers.DockerContainer(container_id, app_info, tool_info, destination_info, job_info, container_description)[source]

Bases: galaxy.tools.deps.containers.Container, galaxy.tools.deps.containers.HasDockerLikeVolumes

container_type = 'docker'
containerize_command(command)[source]

Use destination supplied container configuration parameters, container_id, and command to build a new command that runs input command in container.

class galaxy.tools.deps.containers.HasDockerLikeVolumes[source]

Bases: object

Mixin to share functionality related to Docker volume handling.

Singularity seems to have a fairly compatible syntax for volume handling.

class galaxy.tools.deps.containers.JobInfo(working_directory, tool_directory, job_directory, tmp_directory, job_directory_type)[source]

Bases: object

class galaxy.tools.deps.containers.NullContainer[source]

Bases: object

class galaxy.tools.deps.containers.NullContainerFinder[source]

Bases: object

find_container(tool_info, destination_info, job_info)[source]
class galaxy.tools.deps.containers.SingularityContainer(container_id, app_info, tool_info, destination_info, job_info, container_description)[source]

Bases: galaxy.tools.deps.containers.Container, galaxy.tools.deps.containers.HasDockerLikeVolumes

container_type = 'singularity'
containerize_command(command)[source]

Use destination supplied container configuration parameters, container_id, and command to build a new command that runs input command in container.

class galaxy.tools.deps.containers.ToolInfo(container_descriptions=[], requirements=[], requires_galaxy_python_environment=False, env_pass_through=['GALAXY_SLOTS'])[source]

Bases: object

galaxy.tools.deps.containers.docker_cache_path(cache_directory, container_id)[source]
galaxy.tools.deps.containers.preprocess_volumes(volumes_raw_str, container_type)[source]

Process Galaxy volume specification string to either Docker or Singularity specification.

Galaxy allows the mount try “default_ro” which translates to ro for Docker and ro for Singularity iff no subdirectories are rw (Singularity does not allow ro parent directories with rw subdirectories).

>>> preprocess_volumes("/a/b", DOCKER_CONTAINER_TYPE)
['/a/b:rw']
>>> preprocess_volumes("/a/b:ro,/a/b/c:rw", DOCKER_CONTAINER_TYPE)
['/a/b:ro', '/a/b/c:rw']
>>> preprocess_volumes("/a/b:default_ro,/a/b/c:rw", DOCKER_CONTAINER_TYPE)
['/a/b:ro', '/a/b/c:rw']
>>> preprocess_volumes("/a/b:default_ro,/a/b/c:rw", SINGULARITY_CONTAINER_TYPE)
['/a/b:rw', '/a/b/c:rw']

galaxy.tools.deps.dependencies module

class galaxy.tools.deps.dependencies.DependenciesDescription(requirements=[], installed_tool_dependencies=[])[source]

Bases: object

Capture (in a readily serializable way) context related a tool dependencies - both the tool’s listed requirements and the tool shed related context required to resolve dependencies via the ToolShedPackageDependencyResolver.

This is meant to enable remote resolution of dependencies, by the Pulsar or other potential remote execution mechanisms.

static from_dict(as_dict)[source]
to_dict()[source]

galaxy.tools.deps.docker_util module

Utilities for building up Docker commands…

…using common defaults and configuration mechanisms.

galaxy.tools.deps.docker_util.build_command(image, docker_build_path, **kwds)[source]
galaxy.tools.deps.docker_util.build_docker_cache_command(image, **kwds)[source]
galaxy.tools.deps.docker_util.build_docker_images_command(truncate=True, **kwds)[source]
galaxy.tools.deps.docker_util.build_docker_load_command(**kwds)[source]
galaxy.tools.deps.docker_util.build_docker_run_command(container_command, image, interactive=False, terminal=False, tag=None, volumes=[], volumes_from=None, memory=None, env_directives=[], working_directory=None, name=None, net=None, run_extra_arguments=None, docker_cmd='docker', sudo=True, sudo_cmd='sudo', auto_rm=True, set_user='$UID', host=None)[source]
galaxy.tools.deps.docker_util.build_pull_command(tag, **kwds)[source]
galaxy.tools.deps.docker_util.build_save_image_command(image, destination, **kwds)[source]
galaxy.tools.deps.docker_util.command_list(command, command_args=[], **kwds)[source]

Return Docker command as an argv list.

galaxy.tools.deps.docker_util.command_shell(command, command_args=[], **kwds)[source]

Return Docker command as a string for a shell or command-list.

galaxy.tools.deps.docker_util.kill_command(container, signal=None, **kwds)[source]
galaxy.tools.deps.docker_util.logs_command(container, **kwds)[source]

galaxy.tools.deps.dockerfiles module

galaxy.tools.deps.dockerfiles.docker_host_args(**kwds)[source]
galaxy.tools.deps.dockerfiles.dockerfile_build(path, dockerfile=None, error=<bound method Logger.error of <logging.Logger object>>, **kwds)[source]

galaxy.tools.deps.installable module

Abstractions for installing local software managed and required by Galaxy/galaxy-lib.

class galaxy.tools.deps.installable.InstallableContext[source]

Bases: object

Represent a directory/configuration of something that can be installed.

can_install()[source]

Check preconditions for installation.

installable_description

Short description of thing being installed for log statements.

is_installed()[source]

Return bool indicating if the configured software is installed.

parent_path

Return parent path of the location the installable will be created within.

galaxy.tools.deps.installable.ensure_installed(installable_context, install_func, auto_init)[source]

Make sure target is installed - handle multiple processes potentially attempting installation.

galaxy.tools.deps.requirements module

class galaxy.tools.deps.requirements.ContainerDescription(identifier=None, type='docker', resolve_dependencies=False, shell='/bin/sh')[source]

Bases: object

static from_dict(dict)[source]
to_dict()[source]
class galaxy.tools.deps.requirements.RequirementSpecification(uri, version=None)[source]

Bases: object

Refine a requirement using a URI.

static from_dict(dict)[source]
short_name
specifies_version
to_dict()[source]
class galaxy.tools.deps.requirements.ToolRequirement(name=None, type=None, version=None, specs=[])[source]

Bases: object

Represents an external requirement that must be available for the tool to run (for example, a program, package, or library). Requirements can optionally assert a specific version.

copy()[source]
static from_dict(dict)[source]
to_dict()[source]
class galaxy.tools.deps.requirements.ToolRequirements(tool_requirements=None)[source]

Bases: object

Represents all requirements (packages, env vars) needed to run a tool.

append(requirement)[source]
static from_list(requirements)[source]
packages
resolvable
to_list()[source]
exception galaxy.tools.deps.requirements.ToolRequirementsException[source]

Bases: exceptions.Exception

galaxy.tools.deps.requirements.container_from_element(container_elem)[source]
galaxy.tools.deps.requirements.parse_requirements_from_dict(root_dict)[source]
galaxy.tools.deps.requirements.parse_requirements_from_xml(xml_root)[source]
>>> from xml.etree import ElementTree
>>> def load_requirements( contents ):
...     contents_document = '''<tool><requirements>%s</requirements></tool>'''
...     root = ElementTree.fromstring( contents_document % contents )
...     return parse_requirements_from_xml( root )
>>> reqs, containers = load_requirements('''<requirement>bwa</requirement>''')
>>> reqs[0].name
'bwa'
>>> reqs[0].version is None
True
>>> reqs[0].type
'package'
>>> reqs, containers = load_requirements('''<requirement type="binary" version="1.3.3">cufflinks</requirement>''')
>>> reqs[0].name
'cufflinks'
>>> reqs[0].version
'1.3.3'
>>> reqs[0].type
'binary'

galaxy.tools.deps.singularity_util module

galaxy.tools.deps.singularity_util.build_singularity_run_command(container_command, image, volumes=[], env=[], working_directory=None, singularity_cmd='singularity', run_extra_arguments=None, sudo=False, sudo_cmd='sudo')[source]

galaxy.tools.deps.views module

class galaxy.tools.deps.views.DependencyResolversView(app)[source]

Bases: object

Provide a RESTfulish/JSONy interface to a galaxy.tools.deps.DependencyResolver object. This can be adapted by the Galaxy web framework or other web apps.

clean(index=None, **kwds)[source]
get_requirements_status(tool_requirements_d, installed_tool_dependencies=None)[source]
index()[source]
install_dependencies(requirements)[source]
install_dependency(index=None, **payload)[source]

Installs dependency using highest priority resolver that supports dependency installation (Currently only the conda resolver supports this). If index is given, attempt installation directly using the corresponding resolver. Returns True on success, False on failure. payload is dictionary that must container name, version and type, e.g. {‘name’: ‘numpy’, version=‘1.9.1’, type=’package’}

installable_resolvers

List index for all active resolvers that have the ‘install_dependency’ attribute.

manager_dependency(**kwds)[source]
manager_requirements()[source]
reload()[source]
remove_unused_dependency_paths(envs)[source]

Remove dependencies that are not currently used.

Returns a list of all environments that have been successfully removed.

resolver_dependency(index, **kwds)[source]
resolver_requirements(index)[source]
show(index)[source]
show_dependencies(tool_requirements_d, installed_tool_dependencies=None)[source]

Resolves dependencies to build a requirements status in the admin panel/API

tool_ids_by_requirements

Dictionary with requirements as keys, and tool_ids as values.

toolbox_requirements_status
uninstall_dependencies(index=None, **payload)[source]

Attempt to uninstall requirements. Returns 0 if successfull, else None.

uninstallable_resolvers

List index for all active resolvers that can uninstall dependencies that have been installed through this resolver.

unused_dependency_paths

List dependencies that are not currently installed.

Module contents

Dependency management for tools.

class galaxy.tools.deps.CachedDependencyManager(default_base_path, conf_file=None, app_config={}, tool_dependency_cache_dir=None)[source]

Bases: galaxy.tools.deps.DependencyManager

build_cache(requirements, **kwds)[source]
dependency_shell_commands(requirements, **kwds)[source]

Runs a set of requirements through the dependency resolvers and returns a list of commands required to activate the dependencies. If dependencies are cacheable and the cache does not exist, will try to create it. If cached environment exists or is successfully created, will generate commands to activate it.

get_hashed_dependencies_path(resolved_dependencies)[source]

Returns the path to the hashed dependencies directory (but does not evaluate whether the path exists).

Parameters:resolved_dependencies (list) – list of resolved dependencies
Returns:path
Return type:str
hash_dependencies(resolved_dependencies)[source]

Return hash for dependencies

class galaxy.tools.deps.DependencyManager(default_base_path, conf_file=None, app_config={})[source]

Bases: object

A DependencyManager attempts to resolve named and versioned dependencies by searching for them under a list of directories. Directories should be of the form:

$BASE/name/version/…

and should each contain a file ‘env.sh’ which can be sourced to make the dependency available in the current shell environment.

dependency_shell_commands(requirements, **kwds)[source]
find_dep(name, version=None, type='package', **kwds)[source]
get_app_option(key, default=None)[source]
get_resolver_option(resolver, key, explicit_resolver_options={})[source]

Look in resolver-specific settings for option and then fallback to global settings.

requirements_to_dependencies(requirements, **kwds)[source]

Takes a list of requirements and returns a dictionary with requirements as key and dependencies as value caching these on the tool instance if supplied.

uses_tool_shed_dependencies()[source]
class galaxy.tools.deps.NullDependencyManager[source]

Bases: object

dependency_resolvers = []
dependency_shell_commands(requirements, **kwds)[source]
find_dep(name, version=None, type='package', **kwds)[source]
uses_tool_shed_dependencies()[source]
galaxy.tools.deps.build_dependency_manager(config)[source]