Formatted docs with rstfmt

This commit is contained in:
2025-05-28 17:23:04 +01:00
parent c2902166dc
commit 38ae2fa1c7
8 changed files with 360 additions and 312 deletions

View File

@@ -1,62 +1,66 @@
Development #############
=========== Development
#############
This guide covers setting up a development environment and contributing to ``read_aconity_layers``. This guide covers setting up a development environment and contributing
to ``read_aconity_layers``.
Development Setup *******************
----------------- Development Setup
*******************
Prerequisites Prerequisites
~~~~~~~~~~~~~ =============
* Rust 1.70+ with Cargo - Rust 1.70+ with Cargo
* Python 3.11+ - Python 3.11+
* Poetry - Poetry
* Git - Git
Environment Setup Environment Setup
~~~~~~~~~~~~~~~~~ =================
1. **Clone and setup the repository:** #. **Clone and setup the repository:**
.. code-block:: bash .. code:: bash
git clone https://github.com/Cian-H/read_aconity_layers.git git clone https://github.com/Cian-H/read_aconity_layers.git
cd read_aconity_layers cd read_aconity_layers
2. **Install dependencies:** #. **Install dependencies:**
.. code-block:: bash .. code:: bash
poetry install --with dev,docs poetry install --with dev,docs
3. **Setup pre-commit hooks:** #. **Setup pre-commit hooks:**
.. code-block:: bash .. code:: bash
poetry run pre-commit install poetry run pre-commit install
4. **Build the Rust extension:** #. **Build the Rust extension:**
.. code-block:: bash .. code:: bash
poetry run maturin develop poetry run maturin develop
Code Style and Quality ************************
---------------------- Code Style and Quality
************************
This project uses several tools to maintain code quality: This project uses several tools to maintain code quality:
Python Code Python Code
~~~~~~~~~~~ ===========
* **Ruff**: For linting and formatting - **Ruff**: For linting and formatting
* **MyPy**: For type checking - **MyPy**: For type checking
* **Pytest**: For testing - **Pytest**: For testing
Run quality checks: Run quality checks:
.. code-block:: bash .. code:: bash
# Format and lint Python code # Format and lint Python code
poetry run ruff format . poetry run ruff format .
@@ -69,15 +73,15 @@ Run quality checks:
poetry run pytest poetry run pytest
Rust Code Rust Code
~~~~~~~~~ =========
* **rustfmt**: For formatting - **rustfmt**: For formatting
* **clippy**: For linting - **clippy**: For linting
* **cargo test**: For testing - **cargo test**: For testing
Run quality checks: Run quality checks:
.. code-block:: bash .. code:: bash
# Format Rust code # Format Rust code
cargo fmt cargo fmt
@@ -88,15 +92,17 @@ Run quality checks:
# Run Rust tests # Run Rust tests
cargo test cargo test
Testing *********
------- Testing
*********
The project includes comprehensive tests for both Python and Rust components. The project includes comprehensive tests for both Python and Rust
components.
Running Tests Running Tests
~~~~~~~~~~~~~ =============
.. code-block:: bash .. code:: bash
# Run all tests # Run all tests
poetry run pytest poetry run pytest
@@ -108,32 +114,34 @@ Running Tests
cargo test cargo test
Test Structure Test Structure
~~~~~~~~~~~~~~ ==============
* **Python tests**: Located in ``tests/`` directory - **Python tests**: Located in ``tests/`` directory
* **Rust tests**: Integrated into ``src/rust_fn/mod.rs`` - **Rust tests**: Integrated into ``src/rust_fn/mod.rs``
* **Property-based tests**: Uses ``arbtest`` for Rust property testing - **Property-based tests**: Uses ``arbtest`` for Rust property testing
* **Regression tests**: Validates against known good outputs - **Regression tests**: Validates against known good outputs
Adding Tests Adding Tests
~~~~~~~~~~~~ ============
When adding new functionality: When adding new functionality:
1. **Add Rust tests** in the appropriate module #. **Add Rust tests** in the appropriate module
2. **Add Python integration tests** in ``tests/`` #. **Add Python integration tests** in ``tests/``
3. **Update regression tests** if output format changes #. **Update regression tests** if output format changes
4. **Add property tests** for mathematical functions #. **Add property tests** for mathematical functions
Documentation ***************
------------- Documentation
***************
Building Documentation Building Documentation
~~~~~~~~~~~~~~~~~~~~~~~ ======================
**Prerequisites**: You need the Rust toolchain installed for ``sphinxcontrib-rust`` to work. **Prerequisites**: You need the Rust toolchain installed for
``sphinxcontrib-rust`` to work.
.. code-block:: bash .. code:: bash
# Install documentation dependencies # Install documentation dependencies
poetry install --with docs poetry install --with docs
@@ -149,117 +157,121 @@ Building Documentation
make serve make serve
Documentation Structure Documentation Structure
~~~~~~~~~~~~~~~~~~~~~~~ =======================
* **docs/conf.py**: Sphinx configuration - **docs/conf.py**: Sphinx configuration
* **docs/index.rst**: Main documentation page - **docs/index.rst**: Main documentation page
* **docs/python/**: Python API documentation - **docs/python/**: Python API documentation
* **docs/rust/**: Rust API documentation - **docs/rust/**: Rust API documentation
* **docs/*.rst**: User guides and tutorials - **docs/\*.rst**: User guides and tutorials
The documentation automatically generates API references from: The documentation automatically generates API references from:
* Python docstrings and type hints - Python docstrings and type hints
* Rust documentation comments (``///`` and ``//!``) - Rust documentation comments (``///`` and ``//!``)
* Type stub files (``*.pyi``) - Type stub files (``*.pyi``)
**Note**: For Rust API documentation to work properly, you need: **Note**: For Rust API documentation to work properly, you need:
1. Rust toolchain installed (cargo, rustfmt) #. Rust toolchain installed (cargo, rustfmt)
2. Proper Rust doc comments in your source code #. Proper Rust doc comments in your source code
3. The ``sphinxcontrib-rust`` extension configured correctly #. The ``sphinxcontrib-rust`` extension configured correctly
Contributing **************
------------ Contributing
**************
Workflow Workflow
~~~~~~~~ ========
1. **Fork the repository** on GitHub #. **Fork the repository** on GitHub
2. **Create a feature branch** from ``main`` #. **Create a feature branch** from ``main``
3. **Make your changes** following the coding standards #. **Make your changes** following the coding standards
4. **Add tests** for new functionality #. **Add tests** for new functionality
5. **Update documentation** as needed #. **Update documentation** as needed
6. **Run the full test suite** to ensure everything works #. **Run the full test suite** to ensure everything works
7. **Submit a pull request** #. **Submit a pull request**
Pre-commit Checks Pre-commit Checks
~~~~~~~~~~~~~~~~~ =================
The project uses pre-commit hooks that run automatically: The project uses pre-commit hooks that run automatically:
* Code formatting (Ruff, rustfmt) - Code formatting (Ruff, rustfmt)
* Linting (Ruff, Clippy) - Linting (Ruff, Clippy)
* Type checking (MyPy) - Type checking (MyPy)
* Version bump validation - Version bump validation
* Poetry validation - Poetry validation
These checks must pass before commits are accepted. These checks must pass before commits are accepted.
Release Process Release Process
~~~~~~~~~~~~~~~ ===============
1. **Update version** in ``Cargo.toml`` (triggers version validation) #. **Update version** in ``Cargo.toml`` (triggers version validation)
2. **Update changelog** if applicable #. **Update changelog** if applicable
3. **Ensure all tests pass** #. **Ensure all tests pass**
4. **Create a release** on GitHub #. **Create a release** on GitHub
5. **CI automatically builds and publishes** wheels to PyPI #. **CI automatically builds and publishes** wheels to PyPI
Architecture Notes ********************
------------------ Architecture Notes
********************
The library is structured in two main components: The library is structured in two main components:
Rust Core (``src/rust_fn/``) Rust Core (``src/rust_fn/``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ============================
* **High-performance file I/O** using CSV reader - **High-performance file I/O** using CSV reader
* **Parallel processing** with Rayon - **Parallel processing** with Rayon
* **Memory-efficient array operations** with ndarray - **Memory-efficient array operations** with ndarray
* **Coordinate correction algorithms** - **Coordinate correction algorithms**
Python Bindings (``src/lib.rs``) Python Bindings (``src/lib.rs``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ================================
* **PyO3 integration** for seamless Python interop - **PyO3 integration** for seamless Python interop
* **Error handling** conversion from Rust to Python exceptions - **Error handling** conversion from Rust to Python exceptions
* **NumPy integration** for zero-copy array passing - **NumPy integration** for zero-copy array passing
* **Type annotations** via stub files - **Type annotations** via stub files
Performance Considerations Performance Considerations
~~~~~~~~~~~~~~~~~~~~~~~~~~ ==========================
* File I/O is the primary bottleneck - File I/O is the primary bottleneck
* Parallel processing scales well with core count - Parallel processing scales well with core count
* Memory usage is proportional to dataset size - Memory usage is proportional to dataset size
* Coordinate corrections use vectorized operations - Coordinate corrections use vectorized operations
Common Development Tasks **************************
------------------------ Common Development Tasks
**************************
Adding a New Function Adding a New Function
~~~~~~~~~~~~~~~~~~~~~~ =====================
1. **Implement in Rust** (``src/rust_fn/mod.rs``) #. **Implement in Rust** (``src/rust_fn/mod.rs``)
2. **Add Python binding** (``src/lib.rs``) #. **Add Python binding** (``src/lib.rs``)
3. **Update type stubs** (``read_layers.pyi``) #. **Update type stubs** (``read_layers.pyi``)
4. **Add tests** for both Rust and Python #. **Add tests** for both Rust and Python
5. **Update documentation** #. **Update documentation**
Debugging Build Issues Debugging Build Issues
~~~~~~~~~~~~~~~~~~~~~~ ======================
* **Check Rust version**: Must be 1.70+ - **Check Rust version**: Must be 1.70+
* **Verify PyO3 compatibility**: Should match Python version - **Verify PyO3 compatibility**: Should match Python version
* **Clear build cache**: ``cargo clean`` and ``poetry env remove --all`` - **Clear build cache**: ``cargo clean`` and ``poetry env remove
* **Check dependencies**: Ensure all dev dependencies are installed --all``
- **Check dependencies**: Ensure all dev dependencies are installed
Profiling Performance Profiling Performance
~~~~~~~~~~~~~~~~~~~~~ =====================
For Rust code: For Rust code:
.. code-block:: bash .. code:: bash
# Profile with perf (Linux) # Profile with perf (Linux)
cargo build --release cargo build --release
@@ -268,7 +280,7 @@ For Rust code:
For Python integration: For Python integration:
.. code-block:: bash .. code:: bash
# Profile with py-spy # Profile with py-spy
pip install py-spy pip install py-spy

View File

@@ -12,9 +12,10 @@
.. include:: readme_bottom.rst .. include:: readme_bottom.rst
Indices and tables ####################
================== Indices and tables
####################
* :ref:`genindex` - :ref:`genindex`
* :ref:`modindex` - :ref:`modindex`
* :ref:`search` - :ref:`search`

View File

@@ -1,98 +1,104 @@
Installation ##############
============ Installation
##############
Requirements **************
------------ Requirements
**************
* Python 3.11 or higher - Python 3.11 or higher
* NumPy 2.0.0 or higher - NumPy 2.0.0 or higher
From PyPI (Recommended) *************************
----------------------- From PyPI (Recommended)
*************************
.. code-block:: bash .. code:: bash
pip install read-aconity-layers pip install read-aconity-layers
From Source *************
----------- From Source
*************
If you want to build from source or contribute to the project: If you want to build from source or contribute to the project:
Prerequisites Prerequisites
~~~~~~~~~~~~~ =============
* Rust 1.70 or higher - Rust 1.70 or higher
* Python 3.11 or higher - Python 3.11 or higher
* Poetry (for development) - Poetry (for development)
Build Steps Build Steps
~~~~~~~~~~~ ===========
1. Clone the repository: #. Clone the repository:
.. code-block:: bash .. code:: bash
git clone https://github.com/Cian-H/read_aconity_layers.git git clone https://github.com/Cian-H/read_aconity_layers.git
cd read_aconity_layers cd read_aconity_layers
2. Install Python dependencies: #. Install Python dependencies:
.. code-block:: bash .. code:: bash
poetry install poetry install
3. Build the Rust extension: #. Build the Rust extension:
.. code-block:: bash .. code:: bash
poetry run maturin develop poetry run maturin develop
4. Run tests to verify installation: #. Run tests to verify installation:
.. code-block:: bash .. code:: bash
poetry run pytest poetry run pytest
Development Installation **************************
------------------------ Development Installation
**************************
For development work, you'll also want the development dependencies: For development work, you'll also want the development dependencies:
.. code-block:: bash .. code:: bash
poetry install --with dev,docs poetry install --with dev,docs
This installs additional tools for: This installs additional tools for:
* Code formatting (ruff) - Code formatting (ruff)
* Type checking (mypy) - Type checking (mypy)
* Testing (pytest) - Testing (pytest)
* Documentation building (sphinx) - Documentation building (sphinx)
Troubleshooting *****************
--------------- Troubleshooting
*****************
Common Issues Common Issues
~~~~~~~~~~~~~ =============
**Import Error**: If you get import errors, make sure you've run ``maturin develop`` **Import Error**: If you get import errors, make sure you've run
to build the Rust extension. ``maturin develop`` to build the Rust extension.
**Performance Issues**: The library uses parallel processing by default. If you **Performance Issues**: The library uses parallel processing by default.
encounter memory issues with very large datasets, consider processing files in If you encounter memory issues with very large datasets, consider
smaller batches. processing files in smaller batches.
**Rust Compilation Errors**: Make sure you have a recent version of Rust installed. **Rust Compilation Errors**: Make sure you have a recent version of Rust
The minimum supported version is 1.70. installed. The minimum supported version is 1.70.
Platform Notes Platform Notes
~~~~~~~~~~~~~~ ==============
**Windows**: You may need to install the Microsoft C++ Build Tools if you don't **Windows**: You may need to install the Microsoft C++ Build Tools if
already have them. you don't already have them.
**macOS**: Xcode command line tools are required for Rust compilation. **macOS**: Xcode command line tools are required for Rust compilation.
**Linux**: Most distributions should work out of the box. You may need to install **Linux**: Most distributions should work out of the box. You may need
``build-essential`` on Debian/Ubuntu systems. to install ``build-essential`` on Debian/Ubuntu systems.

View File

@@ -1,25 +1,30 @@
Python API Reference ######################
==================== Python API Reference
######################
This section contains the complete Python API reference for ``read_aconity_layers``. This section contains the complete Python API reference for
``read_aconity_layers``.
Module ********
----------- Module
********
.. automodule:: read_aconity_layers .. automodule:: read_aconity_layers
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
Return Type Information *************************
----------------------- Return Type Information
*************************
The library includes comprehensive type stubs for full IDE support and type checking. The library includes comprehensive type stubs for full IDE support and
type checking.
All functions return NumPy arrays with the following structure: All functions return NumPy arrays with the following structure:
* **Column 0**: X coordinates (corrected) - **Column 0**: X coordinates (corrected)
* **Column 1**: Y coordinates (corrected) - **Column 1**: Y coordinates (corrected)
* **Column 2**: Z coordinates (layer height) - **Column 2**: Z coordinates (layer height)
* **Column 3**: Pyrometer 1 readings - **Column 3**: Pyrometer 1 readings
* **Column 4**: Pyrometer 2 readings - **Column 4**: Pyrometer 2 readings

View File

@@ -1,17 +1,20 @@
Quickstart Guide ##################
================ Quickstart Guide
##################
This guide will get you up and running with ``read_aconity_layers`` in just a few minutes. This guide will get you up and running with ``read_aconity_layers`` in
just a few minutes.
Basic Usage *************
----------- Basic Usage
*************
Reading All Layers from a Directory Reading All Layers from a Directory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ===================================
The most common use case is reading all layer files from a directory: The most common use case is reading all layer files from a directory:
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
import numpy as np import numpy as np
@@ -24,29 +27,25 @@ The most common use case is reading all layer files from a directory:
print(f"Columns: [x, y, z, data1, data2]") print(f"Columns: [x, y, z, data1, data2]")
Reading Specific Files Reading Specific Files
~~~~~~~~~~~~~~~~~~~~~~ ======================
If you want to read only specific layer files: If you want to read only specific layer files:
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
# List of specific files to read # List of specific files to read
files = [ files = ["/path/to/0.1.pcd", "/path/to/0.2.pcd", "/path/to/0.3.pcd"]
"/path/to/0.1.pcd",
"/path/to/0.2.pcd",
"/path/to/0.3.pcd"
]
data = ral.read_selected_layers(files) data = ral.read_selected_layers(files)
Reading a Single Layer Reading a Single Layer
~~~~~~~~~~~~~~~~~~~~~~ ======================
For processing individual layers: For processing individual layers:
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
@@ -58,27 +57,28 @@ For processing individual layers:
y_coords = layer_data[:, 1] y_coords = layer_data[:, 1]
z_coords = layer_data[:, 2] z_coords = layer_data[:, 2]
Working with the Data ***********************
--------------------- Working with the Data
***********************
Understanding the Data Format Understanding the Data Format
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =============================
All functions return NumPy arrays with 5 columns: All functions return NumPy arrays with 5 columns:
* **Column 0**: X coordinates (corrected) - **Column 0**: X coordinates (corrected)
* **Column 1**: Y coordinates (corrected) - **Column 1**: Y coordinates (corrected)
* **Column 2**: Z coordinates (layer height) - **Column 2**: Z coordinates (layer height)
* **Column 3**: Original data column 3 - **Column 3**: Original data column 3
* **Column 4**: Original data column 4 - **Column 4**: Original data column 4
The X and Y coordinates are automatically corrected using the calibration The X and Y coordinates are automatically corrected using the
formulas built into the library. calibration formulas built into the library.
Example: Basic Data Analysis Example: Basic Data Analysis
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ============================
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
import numpy as np import numpy as np
@@ -98,15 +98,15 @@ Example: Basic Data Analysis
plt.figure(figsize=(10, 6)) plt.figure(figsize=(10, 6))
plt.plot(unique_z, layer_counts) plt.plot(unique_z, layer_counts)
plt.xlabel('Layer Height (Z)') plt.xlabel("Layer Height (Z)")
plt.ylabel('Number of Points') plt.ylabel("Number of Points")
plt.title('Points per Layer') plt.title("Points per Layer")
plt.show() plt.show()
Example: Processing by Layer Example: Processing by Layer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ============================
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
import numpy as np import numpy as np
@@ -123,55 +123,59 @@ Example: Processing by Layer
layer_points = data[layer_mask] layer_points = data[layer_mask]
stats = { stats = {
'z': z, "z": z,
'point_count': len(layer_points), "point_count": len(layer_points),
'x_mean': layer_points[:, 0].mean(), "x_mean": layer_points[:, 0].mean(),
'y_mean': layer_points[:, 1].mean(), "y_mean": layer_points[:, 1].mean(),
'data1_mean': layer_points[:, 3].mean(), "data1_mean": layer_points[:, 3].mean(),
'data2_mean': layer_points[:, 4].mean(), "data2_mean": layer_points[:, 4].mean(),
} }
layer_stats.append(stats) layer_stats.append(stats)
# Convert to structured array for easier analysis # Convert to structured array for easier analysis
layer_stats = np.array(layer_stats) layer_stats = np.array(layer_stats)
Performance Tips ******************
---------------- Performance Tips
******************
Parallel Processing Parallel Processing
~~~~~~~~~~~~~~~~~~~ ===================
The library automatically uses parallel processing for multiple files. The library automatically uses parallel processing for multiple files.
For best performance: For best performance:
* Use ``read_layers()`` for directories with many files - Use ``read_layers()`` for directories with many files
* The library will automatically use all available CPU cores - The library will automatically use all available CPU cores
* Larger numbers of files will see better speedup - Larger numbers of files will see better speedup
Memory Usage Memory Usage
~~~~~~~~~~~~ ============
For very large datasets: For very large datasets:
* Consider processing files in batches if memory is limited - Consider processing files in batches if memory is limited
* Use ``read_selected_layers()`` to process subsets - Use ``read_selected_layers()`` to process subsets
* The library streams data efficiently, but the final arrays are held in memory - The library streams data efficiently, but the final arrays are held
in memory
File Organization File Organization
~~~~~~~~~~~~~~~~~ =================
For optimal performance: For optimal performance:
* Keep layer files in a single directory when using ``read_layers()`` - Keep layer files in a single directory when using ``read_layers()``
* Use consistent naming (the Z coordinate is extracted from the filename) - Use consistent naming (the Z coordinate is extracted from the
* Ensure files are properly formatted space-delimited text filename)
- Ensure files are properly formatted space-delimited text
Error Handling ****************
-------------- Error Handling
****************
The library provides detailed error messages for common issues: The library provides detailed error messages for common issues:
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
@@ -182,9 +186,11 @@ The library provides detailed error messages for common issues:
except RuntimeError as e: except RuntimeError as e:
print(f"Processing error: {e}") print(f"Processing error: {e}")
Next Steps ************
---------- Next Steps
************
* Check out the full :doc:`python/index` for detailed function documentation - Check out the full :doc:`python/index` for detailed function
* See :doc:`development` if you want to contribute to the project documentation
* For performance-critical applications, review the :doc:`rust/index` - See :doc:`development` if you want to contribute to the project
- For performance-critical applications, review the :doc:`rust/index`

View File

@@ -1,20 +1,27 @@
Overview ##########
-------- Overview
##########
``read_aconity_layers`` is a high-performance Python library for reading and processing layer data from Aconity mini powder bed fusion machines. It's built with Rust for maximum performance and uses PyO3 for seamless Python integration. ``read_aconity_layers`` is a high-performance Python library for reading
and processing layer data from Aconity mini powder bed fusion machines.
It's built with Rust for maximum performance and uses PyO3 for seamless
Python integration.
Features ##########
-------- Features
##########
* **Fast**: Built with Rust for high-performance data processing - **Fast**: Built with Rust for high-performance data processing
* **Simple**: Easy-to-use Python API - **Simple**: Easy-to-use Python API
* **Parallel**: Leverages Rayon for parallel processing of multiple files - **Parallel**: Leverages Rayon for parallel processing of multiple
* **Type-safe**: Full type annotations and stub files included files
- **Type-safe**: Full type annotations and stub files included
Quick Example ###############
------------- Quick Example
###############
.. code-block:: python .. code:: python
import read_aconity_layers as ral import read_aconity_layers as ral
import numpy as np import numpy as np
@@ -28,4 +35,3 @@ Quick Example
# Read a single layer # Read a single layer
layer = ral.read_layer("/path/to/0.01.pcd") layer = ral.read_layer("/path/to/0.01.pcd")

View File

@@ -1,5 +1,6 @@
read_aconity_layers #####################
================================== read_aconity_layers
#####################
.. image:: https://github.com/Cian-H/read_aconity_layers/workflows/CI/badge.svg .. image:: https://github.com/Cian-H/read_aconity_layers/workflows/CI/badge.svg
:target: https://github.com/Cian-H/read_aconity_layers/actions/workflows/CI.yml :target: https://github.com/Cian-H/read_aconity_layers/actions/workflows/CI.yml
@@ -28,7 +29,7 @@ read_aconity_layers
.. image:: https://img.shields.io/badge/code%20style-Ruff-D7FF64.svg .. image:: https://img.shields.io/badge/code%20style-Ruff-D7FF64.svg
:target: https://github.com/astral-sh/ruff :target: https://github.com/astral-sh/ruff
================================== ----
A library for fast reading of layer data from the aconity mini powder bed fusion machine.
A library for fast reading of layer data from the aconity mini powder
bed fusion machine.

View File

@@ -1,12 +1,15 @@
Rust API Reference ####################
================== Rust API Reference
####################
This section documents the internal Rust implementation of ``read_aconity_layers``. This section documents the internal Rust implementation of
``read_aconity_layers``.
.. note:: .. note::
This documentation is primarily intended for contributors and developers
who want to understand the internal implementation. Most users should This documentation is primarily intended for contributors and
refer to the :doc:`../python/index` instead. developers who want to understand the internal implementation. Most
users should refer to the :doc:`../python/index` instead.
.. toctree:: .. toctree::
:glob: :glob:
@@ -14,44 +17,52 @@ This section documents the internal Rust implementation of ``read_aconity_layers
crates/read_aconity_layers/lib crates/read_aconity_layers/lib
Overview **********
-------- Overview
**********
The Rust implementation provides the high-performance core of ``read_aconity_layers``. The Rust implementation provides the high-performance core of
Key characteristics include: ``read_aconity_layers``. Key characteristics include:
Performance Features Performance Features
~~~~~~~~~~~~~~~~~~~~ ====================
* **Parallel Processing**: Uses Rayon for parallel file reading across all CPU cores - **Parallel Processing**: Uses Rayon for parallel file reading across
* **Memory Efficiency**: Streams data rather than loading everything into memory at once all CPU cores
* **SIMD Operations**: Leverages vectorized operations for coordinate corrections - **Memory Efficiency**: Streams data rather than loading everything
* **Zero-Copy**: Minimizes data copying between Rust and Python using PyO3 into memory at once
- **SIMD Operations**: Leverages vectorized operations for coordinate
corrections
- **Zero-Copy**: Minimizes data copying between Rust and Python using
PyO3
Architecture Architecture
~~~~~~~~~~~~ ============
The crate is organized into two main components: The crate is organized into two main components:
* **Public API** (``src/lib.rs``): PyO3 bindings that expose Rust functions to Python - **Public API** (``src/lib.rs``): PyO3 bindings that expose Rust
* **Core Logic** (``src/rust_fn/``): Pure Rust implementation of file reading and processing functions to Python
- **Core Logic** (``src/rust_fn/``): Pure Rust implementation of file
reading and processing
Error Handling Error Handling
~~~~~~~~~~~~~~ ==============
The Rust code uses a comprehensive ``ReadError`` enum that covers all possible The Rust code uses a comprehensive ``ReadError`` enum that covers all
failure modes, from I/O errors to parsing failures. These are automatically possible failure modes, from I/O errors to parsing failures. These are
converted to appropriate Python exceptions through the PyO3 integration. automatically converted to appropriate Python exceptions through the
PyO3 integration.
Dependencies Dependencies
~~~~~~~~~~~~ ============
Key Rust dependencies that power the performance: Key Rust dependencies that power the performance:
* ``ndarray`` - N-dimensional arrays with BLAS integration - ``ndarray`` - N-dimensional arrays with BLAS integration
* ``rayon`` - Data parallelism library - ``rayon`` - Data parallelism library
* ``csv`` - Fast CSV parsing - ``csv`` - Fast CSV parsing
* ``pyo3`` - Python bindings - ``pyo3`` - Python bindings
* ``numpy`` - NumPy integration for PyO3 - ``numpy`` - NumPy integration for PyO3
* ``glob`` - File path pattern matching - ``glob`` - File path pattern matching
* ``indicatif`` - Progress bars for long operations - ``indicatif`` - Progress bars for long operations