WorkflowConversion.github.io/index.html at master · WorkflowConversion/WorkflowConversion.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
<!DOCTYPE html>
<html>

  <head>
    <meta charset='utf-8'>
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <meta name="description" content="Workflow Conversion : Conversion of workflows across workflow systems">

    <link rel="stylesheet" type="text/css" media="screen" href="stylesheets/stylesheet.css">

    <title>Workflow Conversion</title>
  </head>

  <body>

    <!-- HEADER -->
    <div id="header_wrap" class="outer">
        <header class="inner">
          <a id="forkme_banner" href="https://github.com/WorkflowConversion">View on GitHub</a>

          <h1 id="project_title">Workflow Conversion</h1>
          <h2 id="project_tagline">Conversion of workflows across workflow systems</h2>

        </header>
    </div>

    <!-- MAIN CONTENT -->
    <div id="main_content_wrap" class="outer">
      <section id="main_content" class="inner">
        <h1><a id="why" class="anchor" href="#why" aria-hidden="true"><span class="octicon octicon-link"></span></a>Why?</h1>

<p>We believe that <a href="http://knime.org">KNIME</a> is a great workflow editor. Creating a workflow using <a href="http://knime.org">KNIME</a> entails <i>drag and dropping</i> built-in nodes into a canvas. You can extend <a href="http://knime.org">KNIME</a> by adding nodes developed by the <a href="https://tech.knime.org/community">KNIME community</a> or <a href="https://tech.knime.org/developer/example/extension-wizard">adding your own nodes</a>. Plus, it's <a href="http://www.howtogeek.com/howto/31717/what-do-the-phrases-free-speech-vs.-free-beer-really-mean/">free as in beer!</a></p>

<p>But there's a caveat. <a href="https://www.knime.org/downloads/overview">KNIME Analytics Platform</a> is constrained to the power of the computer on which it runs. This means that if you have a workflow that process lots of data or does extensive number crunching, you're out of luck. <a href="https://www.knime.org/cluster-execution">KNIME Cluster Execution</a> and <a href="https://www.knime.org/knime-server">KNIME Server</a> are royalty-based solutions that enable users to execute workflows on distributed computing interfaces, giving them access to high-performance computing resources. The fact that these solutions are royalty-based shies away potential academic users.</p>

<p>Enter the Grid and Cloud User Support Environment (<a href="https://sourceforge.net/projects/guse">gUSE</a>). <a href="https://sourceforge.net/projects/guse/">gUSE</a> is an open-source, free web-based framework that gives you the ability to execute workflows on distributed computing interfaces. However, creation of workflows is not as fast and user-friendly as it is with <a href="http://knime.org">KNIME</a>.</p>

<p>How can we create workflows in a user-friendly manner on our desktop computers and execute them on a distributed computing interface of our choice?</p>

<p>This is exactly what we strive to do. Convert <a href="http://knime.org">KNIME</a> workflows into <a href="https://sourceforge.net/projects/guse">gUSE</a> workflows. We have taken a great workflow editor and extended it to be able to run workflows on a great workflow framework.</p>

<h1><a id="how" class="anchor" href="#how" aria-hidden="true"><span class="octicon octicon-link"></span></a>How?</h1>

<p>We have been working on a series of projects, each bringing users closer to workflow interoperability. Read through the following sections to obtain detailed information on each of them.</p>

<h2><a id="designer-templates" class="anchor" href="#designer-templates" aria-hidden="true"><span class="octicon octicon-link"></span></a>Common Tool Descriptor</h2>

<p>Common Tool Descriptors (CTD) are XML files that contain all the required information to execute command line tools (i.e., parameters, input files, output files). Using CTDs to wrap your tools is the first step towards workflow interoperability, for CTDs bring a common language to represent individual tools. You can find more information about CTDs <a href="https://github.com/WorkflowConversion/CTDSchema">on their GitHub page</a>.</p>

<p>We offer several bioinformatics suites that are <i>CTD enabled</i>:
	<ul>
		<li><a href="https://github.com/BALL-Project">The Bioinformatics Algorithms Library (BALL)</a></li>
		<li><a href="http://open-ms.sourceforge.net/">Open Mass Spectrometry Framework (OpenMS)</a></li>
		<li><a href="http://www.seqan.de/">The Library for Sequence Analysis (SeqAn)</a></li>
	</ul>
</p>


<h2><a id="designer-templates" class="anchor" href="#designer-templates" aria-hidden="true"><span class="octicon octicon-link"></span></a>Generating CTDs</h2>
<p>If you want to avoid a refactoring of your tools in order to make them <i>CTD enabled</i>, you have a couple of options to generate CTDs that represent your tools:
	<ul>
		<li>Generate a CTD manually - Since CTDs are XML files, you can use your favourite text editor to write your own CTDs. Have a look at the schemas and samples in <a href="https://github.com/WorkflowConversion/CTDSchema">the CTD GitHub page</a>.</li>
		<li>Use <a href="https://github.com/WorkflowConversion/CTDopts">CTDopts</a> - <a href="https://github.com/WorkflowConversion/CTDopts">CTDopts</a> offers a Python script to wrap command line tools to make them <i>CTD enabled</i>. All you need to do is code a few lines of Python code, in which you will <i>describe</i> the parameters of your tool. Once you're done <i>describing</i> the parameters, inputs and outputs, you now have a <i>CTD enabled</i> tool!</li>
	</ul>
</p>

<h2><a id="designer-templates" class="anchor" href="#designer-templates" aria-hidden="true"><span class="octicon octicon-link"></span></a>Taking CTDs a step further</h2>

<p>Once a tool is <i>CTD enabled</i>, you have a couple of options to take it further:</p>

<ul>
	<li><a href="https://github.com/genericworkflownodes/GenericKnimeNodes">Generic KNIME Nodes (GKN)</a> - import your <i>CTD enabled</i> tools into <a href="http://knime.org">KNIME</a>. It is as simple as letting GKN generate a <a href="http://knime.org">KNIME</a> node for you, which you can later import into <a href="http://knime.org">KNIME</a> and make it interact with other nodes. Visit GKN at its <a href="https://github.com/genericworkflownodes/GenericKnimeNodes">GitHub page</a></li>

	<li><a href="https://github.com/WorkflowConversion/CTDConverter">CTDConverter</a> - integrate your <i>CTD enabled</i> tools workflow engines such as in <a href="https://galaxyproject.org/">Galaxy</a> and <a href="https://www.commonwl.org/">CWL</a>. <a href="https://github.com/WorkflowConversion/CTDConverter">CTDConverter</a> generates valid <i>ToolConfigs</i> and <i>CWL</i> stubs that will help you integrate tools. Extending <a href="https://github.com/WorkflowConversion/CTDConverter">CTDConverter</a> to add new supported formats is easy.

</ul>


<h2><a id="designer-templates" class="anchor" href="#designer-templates" aria-hidden="true"><span class="octicon octicon-link"></span></a>KNIME2Grid and the WS-PGRADE extensions</h2>

<p>So you've got yourself a workflow in <a href="http://knime.org">KNIME</a> which you want to export to <a href="https://sourceforge.net/projects/guse/">gUSE</a> in order to execute your workflow on the cloud, a cluster or a grid? No problem! <a href="https://github.com/WorkflowConversion/KNIME2Grid">KNIME2Grid</a>, in conjunction with <a href="https://github.com/WorkflowConversion/WS-PGRADE-Extensions">the WS-PGRADE extensions we developed</a>, lets you export your KNIME workflows to most major batch queueing systems (such as Moab, PBS, SGE) through WS-PGRADE/gUSE.</p>

<p><a href="https://github.com/WorkflowConversion/KNIME2Grid">KNIME2Grid</a> allows you to create and test workflows on KNIME and export them to other workflow engines.Adding supplementary export formats for other workflow engines is as easy as implementing one Java interface!</p>


<h2><a id="designer-templates" class="anchor" href="#designer-templates" aria-hidden="true"><span class="octicon octicon-link"></span></a>Galaxy2gUSE</h2>
<p>If you already have workflows in Galaxy and want to test them on a gUSE instance, you can use <a href="https://github.com/WorkflowConversion/Galaxy2gUSE">Galaxy2gUSE</a>.</p>


      </section>
    </div>

    <!-- FOOTER  -->
    <div id="footer_wrap" class="outer">
      <footer class="inner">
        <p>Published with <a href="https://pages.github.com">GitHub Pages</a></p>
      </footer>
    </div>


  </body>
</html>