feat: support cortex.cpp engine variants

[louis-jan]

Describe Your Changes

This PR aims to integrate the engine management update from cortex.cpp and engine variants.

Where:

The cortex.cpp file now requires the client to set a default variant for engines to run, such as llama-cpp for the mac-arm64 or mac-amd64 platforms.

The folder is structured as follows:

• Jan/data/extensions/@janhq/inference-cortex-extension
  • bin (where engines are downloaded, as before)
    • engines
      • cortex.llamacpp
        • mac-amd64
          • engine_files…
        • mac-arm64
          • engine_files…
      • cortex-server

Cortex extension life-cycle

• Extension Launch -> Boot cortex.cpp -> Health check -> Get System Information -> Set default llama.cpp engine variant

janhq/cortex.cpp#1660

Related Issues

• feat: update inference-cortex-extension to work with the new engine management of cortex.cpp #4025
• bug: Download % suddenly jumped to 100% & back to normal number #3994
• bug: Can't download multiple models at the same time #3995
• bug: Model download process is interrupted but recorded as successful in cortex.db #4024
• bug: [AVX512 GPU] Model fails to start due to missing Cortex engine binary #4033

Code changes

1. Download Script Modifications:

   • Updated download paths in download.sh to use a new directory structure for macOS engine locations, specifically including the version v0.1.35 in the path.

2. Rollup Configurations:

   • Added a new constant CORTEX_ENGINE_VERSION set to 'v0.1.35' in rollup.config.ts.

3. TypeScript Definitions:

   • Declared a new string constant CORTEX_ENGINE_VERSION in global.d.ts.

4. Main Application Logic Enhancements:

   • Added a method setDefaultEngine in index.ts to set the default engine variant on launch.
   • Converted several functions in index.ts to private methods for better encapsulation (healthz, clean, subscribeToEvents).

5. Testing Improvements:

   • Added tests to verify the engineVariant function in execute.test.ts.
   • Generalized enginePath expectations in tests to reflect changes in directory structure.

6. Execution Logic Adjustments:

   • Removed the binPath field from CortexExecutableOptions in execute.ts.
   • Updated the logic for determining the engineVariant based on GPU settings and architecture in execute.ts.

7. Node Module Exports and Utility Functions:

   • Updated the export and utility functions to include engineVariant in both index.ts and engine-executable logic.

These changes seem to focus on organizing engine binaries by version, improving modularity, and enhancing the setup for different system configurations.

[github-actions]

Barecheck - Code coverage report

Total: 69.24%

Your code coverage diff: 0.01% ▴

Uncovered files and lines

File	Lines
web/hooks/useImportModel.ts	46, 65-67, 69, 105, 112-113, 125-126, 131, 133-134
web/screens/Thread/ThreadCenterPanel/LoadModelError/index.tsx	17-21, 23, 25-26, 33, 39, 50-52, 55, 72