


  clmprotocols(5)           FILE FORMATS            clmprotocols(5)


   1. NAME
   2. DESCRIPTION
   3. Network representation
   4. Clustering similarity graphs encoded in BLAST results
   5. Clustering expression data
   6. SEE ALSO
   7. AUTHOR

  NAME
      clmprotocols - Work flows and protocols for mcl and friends

  DESCRIPTION
      A guide to doing analysis with mcl and its helper programs.

  Network representation
      The  clustering  program mcl expects the name of file as its first argu-
      ment.  If the --abc option is used, the file is assumed to adhere  to  a
      simple  format  where  a network is specified edge by edge, one line and
      one edge at a time.  Each line describes an edge as  two  labels  and  a
      numerical  value, all separated by white space. The labels and the value
      respectively identify the two nodes and the edge weight. The  format  is
      called  ABC-format,  where  'A' and 'B' represent the two labels and 'C'
      represents the edge weight. The latter is optional; if omitted the  edge
      weight  is set to one.  If ABC-format is used, the output is returned as
      a listing of clusters, each cluster given as a line of white-space sepa-
      rated labels.

      MCL  can  also utilize a second representation, which is a stringent and
      unambiguous format for both input and output.   This  is  called  matrix
      format  and  it  is required when using other programs in the mcl suite,
      for example when comparing and analysing  clusterings  using  clm(1)  or
      when  extracting  and  transforming  networks using mcx(1).  Native mode
      (matrix format) is entered simply by not specifying --abc.

      The recommended approach using mcl is to convert an external  format  to
      ABC-format. The program mcxload(1) reads the latter and creates a native
      network file and a dictionary file that maps network  nodes  to  labels.
      All  applications  in the MCL suite, including mcl itself, can read this
      native network file format. Label output  can  be  obtained  using  mcx-
      dump(1). The workflow is thus:

         #  External format has been converted to file data.abc (abc format)

         mcxload -abc data.abc --stream-mirror -write-tab data.tab -o data.mci

         mcl data.mci -I 1.4
         mcl data.mci -I 2
         mcl data.mci -I 4

         mcxdump -icl out.data.mci.I14 -tabr data.tab -o dump.data.mci.I14
         mcxdump -icl out.data.mci.I20 -tabr data.tab -o dump.data.mci.I20
         mcxdump -icl out.data.mci.I40 -tabr data.tab -o dump.data.mci.I40

      In this example the cluster output is stored in native format and dumped



  clmprotocols 1.008, 09-308  4 Nov 2009                          1





  clmprotocols(5)           FILE FORMATS            clmprotocols(5)


      to labels using mcxdump. The stored output can now be used to learn more
      about  the  clusterings.  An  example  is the following, where clm(1) is
      applied in mode dist to gauge the distance  between  different  cluster-
      ings.

         clm dist --chain out.data.mci.I{14,20,40}


  Clustering similarity graphs encoded in BLAST results
      A  specific instance of the workflow above is the clustering of proteins
      based on their sequence similarities. In the most typical  scenario  the
      external  format  is  BLAST output, which needs to be transformed to ABC
      format.  In the examples below the input is  in  columnar  blast  format
      obtained  with  the  blast  -m8 option.  It requires a version of mcl at
      least as recent as 09-061.  First we create an ABC-formatted file  using
      the  external  columnar  BLAST  format, which is assumed to be in a file
      called seq.cblast.

         cut -f 1,2,11 seq.cblast > seq.abc

      The columnar format in the file seq.cblast has, for a given  BLAST  hit,
      the sequence labels in the first two columns and the asssociated E-value
      in column 11. It is parsed by the standard UNIX cut(1) utility. The for-
      mat  must have been created with the BLAST -m8 option so that no comment
      lines are present. Alternatively these can be filtered out  using  grep.
      The  newly  created  seq.abc  file is loaded by mcxload(1), which writes
      both a network file seq.mci and a dictionary file seq.tab.

         mcxload -abc seq.abc --stream-mirror --stream-neg-log10 -abc-tf 'ceil(200)'
               -o seq.mci -write-tab seq.tab

      The --stream-mirror option ensures that the resulting  network  will  be
      undirected,  as  recommended  when using mcl. Omitting this option would
      result in a directed network as BLAST E-values generally differ  between
      two sequences. The default course of action for mcxload(1) is to use the
      best value found between a pair of labels. The next  option,  --abc-neg-
      log10  tranforms  the numerical values in the input (the BLAST E-values)
      by taking the logarithm in base 10 and subsequently negating  the  sign.
      Finally,  the  transformed  values  are capped so that any E-value below
      1e-200 is set to a maximum allowed edge weight of 200.

      To obtain clusterings from seq.mci and seq.tab one has two choices.  The
      first is to generate an abstract clustering representation and from that
      obtain the label output, as follows.  Below the -o option is  not  used,
      so  mcl  will  create  meaningful and unique output names by itself. The
      default way of doing this is to preprend the prefix out. and to append a
      suffix  encoding  the inflation value used, with inflation encoded using
      two digits of precision and the decimal separator removed.









  clmprotocols 1.008, 09-308  4 Nov 2009                          2





  clmprotocols(5)           FILE FORMATS            clmprotocols(5)


         mcl seq.mci -I 1.4
         mcl seq.mci -I 2
         mcl seq.mci -I 4
         mcl seq.mci -I 6

         mcxdump -icl out.seq.mci.I14 -tab seq.tab -o dump.seq.mci.I14
         mcxdump -icl out.seq.mci.I20 -tab seq.tab -o dump.seq.mci.I20
         mcxdump -icl out.seq.mci.I40 -tab seq.tab -o dump.seq.mci.I40
         mcxdump -icl out.seq.mci.I60 -tab seq.tab -o dump.seq.mci.I60

      Now the file out.seq.tab.I14 and its associates can be used for  example
      to  compute the distances between the encoded clusterings with clm dist,
      to compute a set of strictly  reconciled  nested  clusterings  with  clm
      order, or to compute an efficiency criterion with clm info.

      Alternatively,  label  output  can be obtained directly from mcl as fol-
      lows.

         mcl seq.mci -I 1.4  -use-tab seq.tab
         mcl seq.mci -I 2  -use-tab seq.tab
         mcl seq.mci -I 4  -use-tab seq.tab
         mcl seq.mci -I 6  -use-tab seq.tab


  Clustering expression data
      The clustering of expression data constitutes another workflow. In  this
      case  the  external  format  usually is a tabular file format containing
      labels for genes or probes and numerical values measuring the expression
      values  or  fold  changes  across a series of conditions or experiments.
      Such  tabular  files  can  be  processed  by  mcxarray(1),  which  comes
      installed with mcl. The program computes correlations (either Pearson or
      Spearmann) between genes, and creates an edge  between  genes  if  their
      correlation  exceeds the specified cutoff. From this mcxarray(1) creates
      both a network file and a dictionary file. In  the  example  below,  the
      file expr.data is in tabular format with one row of column headers (e.g.
      tags for experiments) and one column of row identifiers (e.g.  probe  or
      gene identifiers).

         mcxarray -data expr.data -skipr 1 -skipc 1 -o expr.mci -write-tab expr.tab
            --pearson -co 0.7 -tf 'abs(),add(-0.7)'


      This  uses  the  Pearson  correlation,  ignoring  values below 0.7.  The
      remaining values in the interval [0.7-1] are remapped  to  the  interval
      [0-0.3].  This  is  recommended  so  that  the  edge  weights  will have
      increased contrast between them, as mcl is affected by relative  differ-
      ences (ratios) between edge weights rather than absolute differences. To
      illustrate this, values 0.75 and 0.95 are mapped to 0.05 and 0.25,  with
      respective ratios 0.79 and 0.25.  The network file expr.mci and the dic-
      tionary file expr.tab can now be used as before.

      It is possible to investigate the effect of the  correlation  cutoff  as
      follows.  First a network is generated at a very low threshold, and this
      network is analysed using mcxquery.



  clmprotocols 1.008, 09-308  4 Nov 2009                          3





  clmprotocols(5)           FILE FORMATS            clmprotocols(5)


         mcxarray -data expr.data -skipr 1 -skipc 1 -o expr20.mcx --write-binary
            --pearson -co 0.2 -tf 'abs()'
         mcx query -imx expr20.mcx --vary-correlation


      The output is in a tabular format describing the properties of the  net-
      work  at increasing correlation thresholds. Examples are the size of the
      biggest component, the number of orphan  nodes  (not  connected  to  any
      other node), and the mean and median node degrees.  A good way to choose
      the cutoff is to balance the number of singletons and  the  median  node
      degree.  Both should preferably not be too high.  For example the number
      of orphan nodes should be less than ten percent of the total  number  of
      nodes,  and  the median node degree should be at most one hundred neigh-
      bours.

  SEE ALSO
      mcxio(5).

  AUTHOR
      Stijn van Dongen.





































  clmprotocols 1.008, 09-308  4 Nov 2009                          4


