Skip to content

configuration mechanism - #242

Open
Gitubrr wants to merge 58 commits into
SparseLinearAlgebra:mainfrom
Gitubrr:config
Open

Gitubrr wants to merge 58 commits into
SparseLinearAlgebra:mainfrom
Gitubrr:config

Conversation

@Gitubrr

@Gitubrr Gitubrr commented Apr 25, 2026 •

Copy link
Copy Markdown

Comparison of libraries for creating a configuration mechanism

All information is taken from the repositories of the corresponding projects. Access date: 21.04.26.

Comparison criteria

Criterion Metric Units
Dependencies presence +/-
Sources CLI (argv) / env / files +/-, formats
C++ standard minimum version C++xx
Supports standard hierarchy Linux/Windows/MacOS +/-
Distribution header-only / static / dynamic -
OS Linux / Windows / macOS +/-, compilers
Activity average number of commits commits/month (last month)
Issues open, closed/open n, ratio
Community GitHub Stars n
Passes CI CI +/-

? - not specified in the README.

Comparison of command line parsers (CLI)

Criterion CLI11 cxxopts
Dependencies - -
Sources
 CLI (argv) + +
 env + -
 Files - -
C++ standard ≥ C++11 ≥ C++11
Supports standard hierarchy
 Linux + +
 Windows + +
 MacOS + +
Distribution header-only header-only
OS
 Linux + (GCC 4.8+, Clang 3.5+) + (GCC ≥ 4.9, Clang ≥ 3.1)
 Windows + (MSVC ≥ 2015) + (MSVC ≥ 2015)
 macOS + (AppleClang 7+) + (Clang ≥ 3.1 with libc++)
Community
 Activity (commits/month) 4 1
 Total issues 506 307
 Issues closed/open 9 6
 Stars 4.3k 4.7k
Passes CI + +

File parsers

Criterion nlohmann/json yaml-cpp toml++ toml11 config-cxx taocpp/config
Dependencies - - - - - -
Sources
 CLI (argv) - - - - - -
 env - - - - + +
 Files JSON YAML TOML TOML JSON, YAML, XML JSON, JAXN
C++ standard ≥ C++11 ≥ C++11 ≥ C++17 ≥ C++11 ≥ C++20 ≥ C++17
Supports standard hierarchy
 Linux + + + + + +
 Windows + + + + + +
 MacOS + + + + + +
Distribution header-only static/dynamic¹ header-only/static/dynamic¹ header-only/static/dynamic¹ header-only header-only
OS
 Linux + (GCC 4.8–14.2, Clang 3.4–21.0) + (GCC, Clang) + (Clang 8+, GCC 8+) + (GCC, Clang) + (GCC 13+, Clang 16+) +
 Windows + (MSVC 2015–2022) + (MSVC) + (MSVC VS2019+) + (MSVC, MinGW) + (MSVC 143+ (VS 2022)) +
 macOS + (AppleClang 9.1–16.0) + (Xcode, AppleClang) + (AppleClang) + (AppleClang) + (AppleClang 16+) +
Community
 Activity (commits/month) 2 9 2 4 2 13
 Total issues 3271 905 193 194 24 5
 Issues closed/open 68 3 8 5 0 open 0 open
 Stars 49.4k 15k 2k 1.3k 31 194
Passes CI + + + - + ?

¹ Optional.

Comparison of universal tools

Criterion Boost.Program_options
Dependencies +¹
Sources
 CLI (argv) +
 env +
 Files INI
C++ standard ≥ C++3
Supports standard hierarchy
 Linux +
 Windows +
 MacOS +
Distribution static/dynamic²
OS
 Linux + (GCC 5+, Clang 3.6+)
 Windows + (MSVC 2015 (vc140)+)
 macOS + (AppleClang)
Community
 Activity (commits/month) 12
 Total issues 400
 Issues closed/open 1
 Stars 9.4k
Passes CI +

¹ Dependency list:

  1. Boost.Any
  2. Boost.Bind
  3. Boost.Config
  4. Boost.Core
  5. Boost.Detail
  6. Boost.Function
  7. Boost.Iterator
  8. Boost.Lexical Cast
  9. Boost.Smart Ptr
  10. Boost.ThrowException
  11. Boost.Tokenizer
  12. Boost.Type Traits

² Optional.

Conclusion:

Based on the comparison results, the best choice is likely to use CLI11 and nlohmann/json, since, unlike Boost.program_options, they require no dependencies and are header-only.

Comment thread src/opencl/Compare.md Outdated
"priority": int
"fallback": (???) # Нужен ли fallback

"device":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А почму устройство никак не свзянос оптимизациями? Предположим, я хочу сранвить, как разные параметры влияют на производительность.

Comment thread src/opencl/Compare.md Outdated
Comment thread src/opencl/Compare.md Outdated
@Gitubrr
Gitubrr requested a review from gsvgit May 5, 2026 19:05

@gsvgit gsvgit left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Не вполне опнятне сценарий использования конфига. Кажется, один из возможных такой. В файле (или в нескольких) описываются списки со сконфигурированными устройствами. При зввпуске пользовательского приложения ему передаётся конфиг и (опционально) идентификатор конфигурации, на которой запутиться. Если конфигурация не указана, выбирается дефолтная (решить, как её выбирать).

Comment thread src/opencl/Compare.md Outdated


## Итог:
По результатам сравнения, вероятно, лучшим вариантом будет использование `CLI11` и `nlohmann/json`, так как они, в отличие от `Boost.program_options`, не требуют зависимостей и являются header-only.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А что не так с config-cxx?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Меньше issues и как будто активность поменьше. Я помню, Вы говорнили, что это дискуссионная характеристика. Всетаки config-cxx взять?

Comment thread src/opencl/Compare.md Outdated
Comment thread src/opencl/Compare.md Outdated
config:

"platform":
"selection":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Не понимаю по названию, что это за блок и почему он так назван.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

способ выбора платформы: можно по названию пользователя, по официальному имени и по вендору. Мне кажется, это естественные поля

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

для устройства то же самое

Comment thread src/opencl/Compare.md Outdated
"priority": int
"fallback": (???) # Нужен ли fallback
"device":
"selection":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Такая же история.

Comment thread src/opencl/Compare.md Outdated
"official_name": string
"type": "gpu" | "cpu"
"index": int
"priority": int

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Кажется, про очериди ничего нету.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority не достаточно?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я вот про эти очереди: Количество очередей: сейчас создается только 1 очередь При чём тут priority?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

тогда я не очень понимаю, что в контексте спла значит очередь и какие у нее параметры

@Gitubrr
Gitubrr requested a review from gsvgit May 11, 2026 13:18
Comment thread src/opencl/Compare.md Outdated
allocator_size = 1048576
```

Формата ini.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Откуда тут ini. Ещё и на линуксах. Ещё и в 26 году...

@Gitubrr Gitubrr Jul 10, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Сейчас здесь примеры того, как могут выглядеть конфигурационные файлы в разных форматах. Исходный формат зависит от того, какую библиотеку для парсинга файлов выберем

Кроме этого все впорядке?
Можно уже начать писать сам механизм?
Если комментарии выше больше не актуальны, пометьте, пожалуйста, их как resolved

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

это пример для toml, а не для ini

@Gitubrr
Gitubrr requested a review from gsvgit July 28, 2026 15:26
Comment thread src/opencl/UserGuide.md Outdated
Файл конфигурации, предоставляющийся библиотекой, имеет следующую структуру:
```json
{
"current_config": "default",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange name. Current will be selected by user at the start.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Почему странное? Пользователь может и не выбирать конфигурацию, но чтобы программа не падала этот параметр имеет дефолтное значение. Если пользователь выберет конфигурацию явно, то оно переопределится значением пользователя
Если пользователь не выбрал конфигурацию, то "текущая конфигурация" - дефолтная, если выбрал, "текущая конфигурация" - выбранная пользоватеоем. На мой взгляд название отражает смысл

Comment thread src/opencl/UserGuide.md Outdated
},

"debug_gpu0": {
"extends": ["debug", "gpu0"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can extends be used not for list of configurations, but to redefine fields for particular configuration? As a way to say "I want to set profiling = true for gpu0 config".

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Да.
Можно сделать так:

    "current_config": "gpu_profiling_true",

    "gpu0": { 
        "backend": "by_index", 
        "platform_index": 0, 
        "device_index": 0,
        "if_gpu_unavailable": "abort",
        "queues_count": 1,
        "profiling": false
        "allocator_type": "general",
        "verbosity": 3
        },

    "gpu_profiling_true": { 
        "extends": ["gpu0"],
        "profiling": true
        }

В этом случае параметры конфигурации "gpu0" переопределятся параметрами конфигурации "gpu_profiling_true" ( "profiling": false станет true)

Comment thread src/opencl/UserGuide.md Outdated

"configs": {
"default": {
"backend": "gpu", // переопределен

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? I see only one configuration in the file.

@Gitubrr Gitubrr Oct 5, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Конфигурация "gpu0" включает конфигурацию "default", как базовую. Ее параметры будут переопределены параметрами конфигурации "gpu0". Поэтому "backend": "gpu" переопределяется "backend": "by_index". В extends можно записывать конфигурации, которые находятся в конфигурационных файлах с меньшим приоритетом. (в системном файле пользователь может указать конфигурацию из дефолтного файла, в юзер файле - конфигурацию из системного)
Например:
1 Файл, который идет с библиотекой имеет конфигурацию "default"
2 Файл, созданный пользователем имеет конфигурацию "gpu0"
В поле extends конфигурации "gpu0" пользователь может добавить конфигурацию "default", несмотря на то, что она в дефолтном файле бибилиотеки, а не в файле, который создал пользователь
Это сделано для того, чтобы не нудно было дублировать параметры

@Gitubrr
Gitubrr requested a review from gsvgit October 5, 2026 13:21
Comment thread src/opencl/UserGuide.md Outdated

- name: Run tests
env:
SPLA_CONFIG_FILE: ${{ github.workspace }}/.github/workflows/ci_spla_conf.json

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Нет. Так делать в описываемом сценарии нельзя. Подумайте, почему.

Comment thread .github/workflows/self-hosted.yml Outdated
- name: Check for tests results
run: |
! grep -qi "FAILED" ${{ env.test-file }}
! grep -qE "\[ FAILED \]" ${{ env.test-file }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this changes needed?

Comment thread docs/LibsCompare.md Outdated
@@ -0,0 +1,121 @@
# Сравнение библиотек для создания механизма конфигурирования

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Convert to English

Comment thread docs/UserGuide.md
Comment thread docs/UserManual.md
m_max_wgs = m_device.getInfo<CL_DEVICE_MAX_WORK_GROUP_SIZE>();
m_max_local_mem = m_device.getInfo<CL_DEVICE_LOCAL_MEM_SIZE>();
m_addr_align = m_device.getInfo<CL_DEVICE_MEM_BASE_ADDR_ALIGN>() / 8;// from bits to bytes
m_addr_align = m_device.getInfo<CL_DEVICE_MEM_BASE_ADDR_ALIGN>() / 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why you remove this comment?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They seemed unnecessary. You mentioned that code comments aren't exactly a best practice

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except cases when it is hard to extract some information from code directly.

m_wave_size = 64;
m_is_amd = true;

// Likely, it is an integrated amd device

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this


for (int i = 0; i < count; i++) {
cl_command_queue_properties properties = 0;
#ifndef SPLA_RELEASE

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change needed?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Queue profiling can now be explicitly specified

Comment thread src/opencl/cl_configure.cpp Outdated
->envname("SPLA_PROFILING");

app.add_option("-sa,--spla-allocator-type", cfg.allocator_type,
"Allocator type:\n"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yet another description of flags/ Why?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files src/opencl/cl_configure.cpp and src/opencl/cl_configure.hpp were moved to src/core, but I forgot to delete these ones

Comment thread src/core/logger.hpp
MessageCallback m_callback;

MessageCallback m_callback;
int m_verbosity = 3;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Comment thread docs/LibsCompare.md
@@ -0,0 +1,121 @@
# Comparison of libraries for creating a configuration mechanism

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to PR description.

Comment thread python/pyspla/bridge.py
print(f"pyspla: [{decoded_file}:{line}] {_status_mapping[status]}: {decoded_msg}")


def filter_spla_args(argv):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use standard Python any package for arguments processing?

Comment thread src/binding/c_library.cpp
auto* lib = spla::Library::get();
if (!lib) return SPLA_STATUS_ERROR;

if (spla::config_final.backend.has_value() &&

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What this condition means? Why you check manually only this condition?

LOG_MSG(Status::DeviceNotFound, "index out of list of available devices");
return Status::DeviceNotFound;
}
m_platform.getDevices(CL_DEVICE_TYPE_ALL, &available_devices);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we request all types always instead of GPU?

m_max_wgs = m_device.getInfo<CL_DEVICE_MAX_WORK_GROUP_SIZE>();
m_max_local_mem = m_device.getInfo<CL_DEVICE_LOCAL_MEM_SIZE>();
m_addr_align = m_device.getInfo<CL_DEVICE_MEM_BASE_ADDR_ALIGN>() / 8;// from bits to bytes
m_addr_align = m_device.getInfo<CL_DEVICE_MEM_BASE_ADDR_ALIGN>() / 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except cases when it is hard to extract some information from code directly.

Comment thread tools/soft_check.cpp
return spla_Configure(static_cast<int>(args.size()), args.data());
}

void print_usage() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really? C++ has no library that can automatically infer usage from registered CLI arguments?

Comment thread CMakeLists.txt

if (SPLA_BUILD_OPENCL)
if (SPLA_TARGET_MACOSX)
message(STATUS "Add standard Apple OpenCL package")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these changes required?

Comment thread CMakeLists.txt

add_library(OpenCLHeaders INTERFACE)
add_library(OpenCL::Headers ALIAS OpenCLHeaders)
target_include_directories(OpenCLHeaders INTERFACE ${OpenCL_INCLUDE_DIRS})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this.

Comment thread CMakeLists.txt
endif ()


message(STATUS "Add Khronos OpenCL HPP bindings")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this.

Comment thread CMakeLists.txt
if (SPLA_BUILD_OPENCL)
target_link_libraries(spla PUBLIC OpenCL)
target_link_libraries(spla PUBLIC OpenCL::HeadersCpp)
target_compile_definitions(spla PUBLIC CL_TARGET_OPENCL_VERSION=120)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What the reason of these changes?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants