rocALUTION Documentation

rocALUTION is a sparse linear algebra library with focus on exploring fine-grained parallelism on top of AMD’s Radeon Open Compute ROCm runtime and toolchains, targeting modern CPU and GPU platforms. Based on C++ and HIP, it provides a portable, generic and flexible design that allows seamless integration with other scientific software packages.

In the following, three separate chapters are available:

• User Manual: This is the manual of rocALUTION. It can be seen as a starting guide for new users but also a reference book for more experienced users.

• Design Documentation: The Design Document is targeted to advanced users / developers that want to understand, modify or extend the functionality of the rocALUTION library. To embed rocALUTION into your project, it is not required to read the Design Document.

• API: This is a list of API functions provided by rocALUTION.

Contents:

• Design Documentation
  • Design and Philosophy
  • Library Source Code Organization
    • Library Source Code Organization
      • The src/base/ directory
      • The src/solvers/ directory
      • The src/utils/ directory
  • Functionality Extension Guidelines
    • LocalMatrix Functionlity Extension
      • API Enhancement
      • Enhancement of the BaseMatrix class
    • Adding a Solver
      • API Enhancement
  • Functionality Table
    • LocalMatrix and LocalVector classes
    • Solver and Preconditioner classes
  • Clients
    • Examples
    • Unit Tests
• User Manual
  • Introduction
    • Overview
    • License
  • Building and Installing
    • Installing from AMD ROCm repository
    • Building from GitHub repository
      • Download rocALUTION
      • Using install.sh script to build rocALUTION with dependencies
      • Using install.sh script to build rocALUTION with dependencies and clients
      • Using individual commands to build rocALUTION
      • Common build problems
      • Simple Test
  • Basics
    • Operators and Vectors
      • Local Operators and Vectors
      • Global Operators and Vectors
    • Backend Descriptor and User Control
      • Initialization of rocALUTION
      • Thread-core Mapping
      • OpenMP Threshold Size
      • Accelerator Selection
      • Disable the Accelerator
      • Backend Information
      • MPI and Multi-Accelerators
    • Automatic Object Tracking
    • Verbose Output
    • Verbose Output and MPI
    • Debug Output
    • File Logging
    • Versions
  • Single-node Computation
    • Introduction
    • ValueType
    • Complex Support
    • Allocation and Free
      • Allocate()
      • Clear()
      • AllocateCOO()
      • AllocateCSR()
      • AllocateBCSR()
      • AllocateMCSR()
      • AllocateELL()
      • AllocateDIA()
      • AllocateHYB()
      • AllocateDENSE()
      • Clear()
    • Matrix Formats
      • COO storage format
      • CSR storage format
      • BCSR storage format
      • ELL storage format
      • DIA storage format
      • HYB storage format
      • Memory Usage
    • File I/O
      • ReadFileASCII()
      • WriteFileASCII()
      • ReadFileBinary()
      • WriteFileBinary()
      • ReadFileMTX()
      • WriteFileMTX()
      • ReadFileCSR()
      • WriteFileCSR()
    • Access
      • operator[]()
      • operator[]()
    • Raw Access to the Data
      • SetDataPtr
      • LeaveDataPtr
    • Copy CSR Matrix Host Data
      • CopyFromHostCSR()
    • Copy Data
      • CopyFromData()
      • CopyToData()
    • Object Info
      • Info()
    • Copy
      • CopyFrom()
      • CopyFrom()
      • CopyFrom()
    • Clone
      • CloneFrom
      • CloneBackend
    • Check
      • Check()
      • Check()
    • Sort
      • Sort()
    • Keying
      • Key()
    • Graph Analyzers
      • Cuthill-McKee Ordering
      • Maximal Independent Set
      • Multi-Coloring
      • Zero Block Permutation
      • Connectivity Ordering
    • Basic Linear Algebra Operations
  • Multi-node Computation
    • Introduction
    • Code Structure
    • Parallel Manager
      • rocalution::ParallelManager
      • SetMPICommunicator()
      • Clear()
      • GetGlobalSize()
      • GetLocalSize()
      • GetNumReceivers()
      • GetNumSenders()
      • GetNumProcs()
      • SetGlobalSize()
      • SetLocalSize()
      • SetBoundaryIndex()
      • SetReceivers()
      • SetSenders()
      • ReadFileASCII()
      • WriteFileASCII()
    • Global Matrices and Vectors
      • GetInterior()
      • GetGhost()
      • Asynchronous SpMV
    • File I/O
      • File Organization
      • Parallel Manager
      • Matrices
      • Vectors
  • Solvers
    • Code Structure
      • rocalution::Solver
      • SetOperator()
      • Build()
      • Clear()
      • Solve()
      • Print()
      • ReBuildNumeric()
      • MoveToHost()
      • MoveToAccelerator()
    • Iterative Linear Solvers
      • rocalution::IterativeLinearSolver
      • Init()
      • Init()
      • InitMinIter()
      • InitMaxIter()
      • InitTol()
      • RecordResidualHistory()
      • RecordHistory()
      • Verbose()
      • SetPreconditioner()
      • SetResidualNorm()
      • GetAmaxResidualIndex()
      • GetSolverStatus()
    • Building and Solving Phase
    • Clear Function and Destructor
    • Numerical Update
    • Fixed-Point Iteration
      • rocalution::FixedPoint
      • SetRelaxation()
    • Krylov Subspace Solvers
      • CG
      • CR
      • GMRES
      • FGMRES
      • BiCGStab
      • IDR
      • FCG
      • QMRCGStab
      • BiCGStab(l)
    • Chebyshev Iteration Scheme
      • rocalution::Chebyshev
    • Mixed-Precision Defect Correction Scheme
      • rocalution::MixedPrecisionDC
    • MultiGrid Solvers
      • rocalution::BaseMultiGrid
      • Geometric MultiGrid
      • Algebraic MultiGrid
    • Unsmoothed Aggregation AMG
      • rocalution::UAAMG
      • SetCouplingStrength()
      • SetOverInterp()
    • Smoothed Aggregation AMG
      • rocalution::SAAMG
      • SetCouplingStrength()
      • SetInterpRelax()
    • Ruge-Stueben AMG
      • rocalution::RugeStuebenAMG
      • SetCouplingStrength()
    • Pairwise AMG
      • rocalution::PairwiseAMG
      • SetBeta()
      • SetOrdering()
      • SetCoarseningFactor()
    • Direct Linear Solvers
      • rocalution::DirectLinearSolver
      • rocalution::LU
      • rocalution::QR
      • rocalution::Inversion
  • Preconditioners
    • rocalution::Preconditioner
    • Code Structure
    • Jacobi Method
      • rocalution::Jacobi
    • (Symmetric) Gauss-Seidel / (S)SOR Method
      • rocalution::GS
      • rocalution::SGS
    • Incomplete Factorizations
      • ILU
      • ILUT
      • IC
    • AI Chebyshev
      • rocalution::AIChebyshev
      • Set()
    • FSAI
      • rocalution::FSAI
      • Set()
      • Set()
      • SetPrecondMatrixFormat()
    • SPAI
      • rocalution::SPAI
      • SetPrecondMatrixFormat()
    • TNS
      • rocalution::TNS
      • Set()
      • SetPrecondMatrixFormat()
    • MultiColored Preconditioners
      • rocalution::MultiColored
      • SetPrecondMatrixFormat()
      • SetDecomposition()
      • MultiColored (Symmetric) Gauss-Seidel / (S)SOR
      • MultiColored Power(q)-pattern method ILU(p,q)
    • Multi-Elimination Incomplete LU
      • rocalution::MultiElimination
      • GetSizeDiagBlock()
      • GetLevel()
      • Set()
      • SetPrecondMatrixFormat()
    • Diagonal Preconditioner for Saddle-Point Problems
      • rocalution::DiagJacobiSaddlePointPrecond
      • Set()
    • (Restricted) Additive Schwarz Preconditioner
      • rocalution::AS
      • Set()
      • rocalution::RAS
    • Block-Jacobi (MPI) Preconditioner
      • rocalution::BlockJacobi
      • Set()
    • Block Preconditioner
      • rocalution::BlockPreconditioner
      • Set()
      • SetDiagonalSolver()
      • SetLSolver()
      • SetExternalLastMatrix()
      • SetPermutation()
    • Variable Preconditioner
      • rocalution::VariablePreconditioner
      • SetPreconditioner()
  • Backends
    • Moving Objects To and From the Accelerator
      • MoveToAccelerator()
      • MoveToHost()
    • Asynchronous Transfers
      • MoveToAcceleratorAsync()
      • MoveToHostAsync()
      • Sync()
    • Systems without Accelerators
    • Memory Allocations
      • Allocation Problems
      • Memory Alignment
      • Pinned Memory Allocation (HIP)
  • Remarks
    • Performance
    • Accelerators
    • Correctness
  • Supported Targets
• API
  • Host Utility Functions
    • allocate_host()
    • free_host()
    • set_to_zero_host()
    • rocalution_time()
  • Backend Manager
    • init_rocalution()
    • stop_rocalution()
    • set_device_rocalution()
    • set_omp_threads_rocalution()
    • set_omp_affinity_rocalution()
    • set_omp_threshold_rocalution()
    • info_rocalution()
    • info_rocalution()
    • disable_accelerator_rocalution()
    • _rocalution_sync()
  • Base Rocalution
    • rocalution::BaseRocalution
      • MoveToAccelerator()
      • MoveToHost()
      • MoveToAcceleratorAsync()
      • MoveToHostAsync()
      • Sync()
      • CloneBackend()
      • Info()
      • Clear()
  • Operator
    • rocalution::Operator
      • GetM()
      • GetN()
      • GetNnz()
      • GetLocalM()
      • GetLocalN()
      • GetLocalNnz()
      • GetGhostM()
      • GetGhostN()
      • GetGhostNnz()
      • Apply()
      • ApplyAdd()
      • Apply()
      • ApplyAdd()
  • Vector
    • rocalution::Vector
      • CopyFrom()
      • CopyFrom()
      • CloneFrom()
      • CloneFrom()
      • GetSize()
      • GetLocalSize()
      • GetGhostSize()
      • Check()
      • Clear()
      • Zeros()
      • Ones()
      • SetValues()
      • SetRandomUniform()
      • SetRandomNormal()
      • ReadFileASCII()
      • WriteFileASCII()
      • ReadFileBinary()
      • WriteFileBinary()
      • CopyFromAsync()
      • CopyFromFloat()
      • CopyFromDouble()
      • CopyFrom()
      • AddScale()
      • AddScale()
      • ScaleAdd()
      • ScaleAdd()
      • ScaleAddScale()
      • ScaleAddScale()
      • ScaleAddScale()
      • ScaleAddScale()
      • ScaleAdd2()
      • ScaleAdd2()
      • Scale()
      • Dot()
      • Dot()
      • DotNonConj()
      • DotNonConj()
      • Norm()
      • Reduce()
      • Asum()
      • Amax()
      • PointWiseMult()
      • PointWiseMult()
      • PointWiseMult()
      • PointWiseMult()
      • Power()
  • Local Matrix
    • rocalution::LocalMatrix
      • AllocateCSR()
      • AllocateBCSR()
      • AllocateMCSR()
      • AllocateCOO()
      • AllocateDIA()
      • AllocateELL()
      • AllocateHYB()
      • AllocateDENSE()
      • SetDataPtrCOO()
      • SetDataPtrCSR()
      • SetDataPtrBCSR()
      • SetDataPtrMCSR()
      • SetDataPtrELL()
      • SetDataPtrDIA()
      • SetDataPtrDENSE()
      • LeaveDataPtrCOO()
      • LeaveDataPtrCSR()
      • LeaveDataPtrBCSR()
      • LeaveDataPtrMCSR()
      • LeaveDataPtrELL()
      • LeaveDataPtrDIA()
      • LeaveDataPtrDENSE()
      • Info()
      • GetFormat()
      • GetBlockDimension()
      • GetM()
      • GetN()
      • GetNnz()
      • Check()
      • Clear()
      • Zeros()
      • Scale()
      • ScaleDiagonal()
      • ScaleOffDiagonal()
      • AddScalar()
      • AddScalarDiagonal()
      • AddScalarOffDiagonal()
      • ExtractSubMatrix()
      • ExtractSubMatrices()
      • ExtractDiagonal()
      • ExtractInverseDiagonal()
      • ExtractU()
      • ExtractL()
      • Permute()
      • PermuteBackward()
      • CMK()
      • RCMK()
      • ConnectivityOrder()
      • MultiColoring()
      • MaximalIndependentSet()
      • ZeroBlockPermutation()
      • ILU0Factorize()
      • LUFactorize()
      • ILUTFactorize()
      • ILUpFactorize()
      • LUAnalyse()
      • LUAnalyseClear()
      • LUSolve()
      • ICFactorize()
      • LLAnalyse()
      • LLAnalyseClear()
      • LLSolve()
      • LLSolve()
      • LAnalyse()
      • LAnalyseClear()
      • LSolve()
      • UAnalyse()
      • UAnalyseClear()
      • USolve()
      • Householder()
      • QRDecompose()
      • QRSolve()
      • Invert()
      • ReadFileMTX()
      • WriteFileMTX()
      • ReadFileCSR()
      • WriteFileCSR()
      • MoveToAccelerator()
      • MoveToAcceleratorAsync()
      • MoveToHost()
      • MoveToHostAsync()
      • Sync()
      • CopyFrom()
      • CopyFromAsync()
      • CloneFrom()
      • UpdateValuesCSR()
      • CopyFromCSR()
      • CopyToCSR()
      • CopyFromCOO()
      • CopyToCOO()
      • CopyFromHostCSR()
      • CreateFromMap()
      • CreateFromMap()
      • ConvertToCSR()
      • ConvertToMCSR()
      • ConvertToBCSR()
      • ConvertToCOO()
      • ConvertToELL()
      • ConvertToDIA()
      • ConvertToHYB()
      • ConvertToDENSE()
      • ConvertTo()
      • Apply()
      • ApplyAdd()
      • SymbolicPower()
      • MatrixAdd()
      • MatrixMult()
      • DiagonalMatrixMult()
      • DiagonalMatrixMultL()
      • DiagonalMatrixMultR()
      • Gershgorin()
      • Compress()
      • Transpose()
      • Transpose()
      • Sort()
      • Key()
      • ReplaceColumnVector()
      • ReplaceRowVector()
      • ExtractColumnVector()
      • ExtractRowVector()
      • AMGConnect()
      • AMGAggregate()
      • AMGPMISAggregate()
      • AMGSmoothedAggregation()
      • AMGSmoothedAggregation()
      • AMGAggregation()
      • AMGAggregation()
      • RSCoarsening()
      • RSPMISCoarsening()
      • RSDirectInterpolation()
      • RSDirectInterpolation()
      • RSExtPIInterpolation()
      • RSExtPIInterpolation()
      • RugeStueben()
      • FSAI()
      • SPAI()
      • InitialPairwiseAggregation()
      • InitialPairwiseAggregation()
      • FurtherPairwiseAggregation()
      • FurtherPairwiseAggregation()
      • CoarsenOperator()
  • Local Stencil
    • rocalution::LocalStencil
      • LocalStencil()
      • Info()
      • GetNDim()
      • GetM()
      • GetN()
      • GetNnz()
      • SetGrid()
      • Clear()
      • Apply()
      • ApplyAdd()
      • MoveToAccelerator()
      • MoveToHost()
  • Global Matrix
    • rocalution::GlobalMatrix
      • GlobalMatrix()
      • GetM()
      • GetN()
      • GetNnz()
      • GetLocalM()
      • GetLocalN()
      • GetLocalNnz()
      • GetGhostM()
      • GetGhostN()
      • GetGhostNnz()
      • MoveToAccelerator()
      • MoveToHost()
      • Info()
      • Check()
      • AllocateCSR()
      • AllocateCOO()
      • Clear()
      • SetParallelManager()
      • SetDataPtrCSR()
      • SetDataPtrCOO()
      • SetLocalDataPtrCSR()
      • SetLocalDataPtrCOO()
      • SetGhostDataPtrCSR()
      • SetGhostDataPtrCOO()
      • LeaveDataPtrCSR()
      • LeaveDataPtrCOO()
      • LeaveLocalDataPtrCSR()
      • LeaveLocalDataPtrCOO()
      • LeaveGhostDataPtrCSR()
      • LeaveGhostDataPtrCOO()
      • CloneFrom()
      • CopyFrom()
      • ConvertToCSR()
      • ConvertToMCSR()
      • ConvertToBCSR()
      • ConvertToCOO()
      • ConvertToELL()
      • ConvertToDIA()
      • ConvertToHYB()
      • ConvertToDENSE()
      • ConvertTo()
      • Apply()
      • ApplyAdd()
      • ReadFileMTX()
      • WriteFileMTX()
      • ReadFileCSR()
      • WriteFileCSR()
      • Sort()
      • ExtractInverseDiagonal()
      • Scale()
      • InitialPairwiseAggregation()
      • FurtherPairwiseAggregation()
      • CoarsenOperator()
  • Local Vector
    • rocalution::LocalVector
      • operator[]()
      • operator[]()
      • MoveToAccelerator()
      • MoveToAcceleratorAsync()
      • MoveToHost()
      • MoveToHostAsync()
      • Sync()
      • Info()
      • GetSize()
      • Check()
      • Allocate()
      • SetDataPtr()
      • LeaveDataPtr()
      • Clear()
      • Zeros()
      • Ones()
      • SetValues()
      • SetRandomUniform()
      • SetRandomNormal()
      • ReadFileASCII()
      • WriteFileASCII()
      • ReadFileBinary()
      • WriteFileBinary()
      • CopyFrom()
      • CopyFromAsync()
      • CopyFromFloat()
      • CopyFromDouble()
      • CopyFrom()
      • CopyFromPermute()
      • CopyFromPermuteBackward()
      • CloneFrom()
      • CopyFromData()
      • CopyToData()
      • Permute()
      • PermuteBackward()
      • Restriction()
      • Prolongation()
      • AddScale()
      • ScaleAdd()
      • ScaleAddScale()
      • ScaleAddScale()
      • ScaleAdd2()
      • Scale()
      • Dot()
      • DotNonConj()
      • Norm()
      • Reduce()
      • Asum()
      • Amax()
      • PointWiseMult()
      • PointWiseMult()
      • Power()
      • SetIndexArray()
      • GetIndexValues()
      • GetIndexValues()
      • SetIndexValues()
      • SetIndexValues()
      • GetContinuousValues()
      • SetContinuousValues()
      • ExtractCoarseMapping()
      • ExtractCoarseBoundary()
  • Global Vector
    • rocalution::GlobalVector
      • GlobalVector()
      • MoveToAccelerator()
      • MoveToHost()
      • Info()
      • Check()
      • GetSize()
      • GetLocalSize()
      • GetGhostSize()
      • Allocate()
      • Clear()
      • SetParallelManager()
      • Zeros()
      • Ones()
      • SetValues()
      • SetRandomUniform()
      • SetRandomNormal()
      • CloneFrom()
      • operator[]()
      • operator[]()
      • SetDataPtr()
      • LeaveDataPtr()
      • CopyFrom()
      • ReadFileASCII()
      • WriteFileASCII()
      • ReadFileBinary()
      • WriteFileBinary()
      • AddScale()
      • ScaleAdd()
      • ScaleAdd2()
      • ScaleAddScale()
      • Scale()
      • Dot()
      • DotNonConj()
      • Norm()
      • Reduce()
      • Asum()
      • Amax()
      • PointWiseMult()
      • PointWiseMult()
      • Power()
      • Restriction()
      • Prolongation()
  • Base Classes
    • rocalution::BaseMatrix
    • rocalution::BaseStencil
    • rocalution::BaseVector
    • rocalution::HostMatrix
    • rocalution::HostStencil
    • rocalution::HostVector
    • rocalution::AcceleratorMatrix
    • rocalution::AcceleratorStencil
    • rocalution::AcceleratorVector
  • Parallel Manager
    • rocalution::ParallelManager
      • SetMPICommunicator()
      • Clear()
      • GetRank()
      • GetGlobalSize()
      • GetGlobalNrow()
      • GetGlobalNcol()
      • GetLocalSize()
      • GetLocalNrow()
      • GetLocalNcol()
      • GetNumReceivers()
      • GetNumSenders()
      • GetNumProcs()
      • SetGlobalSize()
      • SetGlobalNrow()
      • SetGlobalNcol()
      • SetLocalSize()
      • SetLocalNrow()
      • SetLocalNcol()
      • SetBoundaryIndex()
      • SetReceivers()
      • SetSenders()
      • LocalToGlobal()
      • GlobalToLocal()
      • Status()
      • ReadFileASCII()
      • WriteFileASCII()
  • Solvers
    • rocalution::Solver
      • SetOperator()
      • ResetOperator()
      • Print()
      • Solve()
      • SolveZeroSol()
      • Clear()
      • Build()
      • BuildMoveToAcceleratorAsync()
      • Sync()
      • ReBuildNumeric()
      • MoveToHost()
      • MoveToAccelerator()
      • Verbose()
    • Iterative Linear Solvers
      • rocalution::IterativeLinearSolver
      • rocalution::FixedPoint
      • rocalution::MixedPrecisionDC
      • rocalution::Chebyshev
      • Krylov Subspace Solvers
      • MultiGrid Solvers
    • Direct Solvers
      • rocalution::DirectLinearSolver
      • rocalution::Inversion
      • rocalution::LU
      • rocalution::QR
  • Preconditioners
    • rocalution::Preconditioner
      • SolveZeroSol()
    • rocalution::AIChebyshev
      • Print()
      • Solve()
      • Set()
      • Build()
      • Clear()
    • rocalution::FSAI
      • Print()
      • Solve()
      • Set()
      • Set()
      • Build()
      • Clear()
      • SetPrecondMatrixFormat()
    • rocalution::SPAI
      • Print()
      • Solve()
      • Build()
      • Clear()
      • SetPrecondMatrixFormat()
    • rocalution::TNS
      • Print()
      • Set()
      • Solve()
      • Build()
      • Clear()
      • SetPrecondMatrixFormat()
    • rocalution::AS
      • Print()
      • Set()
      • Solve()
      • Build()
      • Clear()
    • rocalution::RAS
      • Print()
      • Solve()
    • rocalution::BlockJacobi
      • Print()
      • Set()
      • Solve()
      • SolveZeroSol()
      • Build()
      • ReBuildNumeric()
      • Clear()
    • rocalution::BlockPreconditioner
      • Print()
      • Clear()
      • Set()
      • SetDiagonalSolver()
      • SetLSolver()
      • SetExternalLastMatrix()
      • SetPermutation()
      • Build()
      • Solve()
    • rocalution::Jacobi
      • Print()
      • Solve()
      • Build()
      • Clear()
      • ResetOperator()
    • rocalution::GS
      • Print()
      • Solve()
      • Build()
      • Clear()
      • ResetOperator()
    • rocalution::SGS
      • Print()
      • Solve()
      • Build()
      • Clear()
      • ResetOperator()
    • rocalution::ILU
      • Print()
      • Solve()
      • Set()
      • Build()
      • Clear()
    • rocalution::ILUT
      • Print()
      • Solve()
      • Set()
      • Set()
      • Build()
      • Clear()
    • rocalution::IC
      • Print()
      • Solve()
      • Build()
      • Clear()
    • rocalution::VariablePreconditioner
      • Print()
      • Solve()
      • Build()
      • Clear()
      • SetPreconditioner()
    • rocalution::MultiColored
      • Clear()
      • Build()
      • SetPrecondMatrixFormat()
      • SetDecomposition()
      • Solve()
    • rocalution::MultiColoredSGS
      • Print()
      • ReBuildNumeric()
      • SetRelaxation()
    • rocalution::MultiColoredGS
      • Print()
    • rocalution::MultiColoredILU
      • Print()
      • ReBuildNumeric()
      • Set()
      • Set()
    • rocalution::MultiElimination
      • GetSizeDiagBlock()
      • GetLevel()
      • Print()
      • Clear()
      • Set()
      • SetPrecondMatrixFormat()
      • Build()
      • Solve()
    • rocalution::DiagJacobiSaddlePointPrecond
      • Print()
      • Clear()
      • Set()
      • Build()
      • Solve()