Этот сайт использует файлы cookies. Продолжая просмотр страниц сайта, вы соглашаетесь с использованием файлов cookies. Если вам нужна дополнительная информация, пожалуйста, посетите страницу Политика файлов Cookie
Subscribe
Прямой эфир
Русский Русский  English English
Войти / Регистрация
Cryptocurrencies: 8123 / Markets: 110688
Market Cap: $ 2 690 667 610 546 / 24h Vol: $ 60 048 265 646 / BTC Dominance: 60.138218143661%

Н Новости

  1. Главная
  2. Новости

Библиотека JIT-компиляции Loops для оптимизации нейросетей, СУБД и не только

Loop fusion is a compiler transformation in which two adjacent loops are merged into a single loop over the same index range. This transformation is typically applied to reduce loop overhead and improve run-time performance.

— Intel compiler guide

Привет, Хабр! Меня зовут Пётр Чекмарёв, я старший инженер компании YADRO, занимаюсь компьютерным зрением на мобильных устройствах и низкоуровневой оптимизацией плотных вычислительных функций.

Оптимизация кода — вечная тема, особенно актуальная в дни триумфального шествия искусственного интеллекта. Оптимально написанные, но изолированные ядра сетей составляются в разные последовательности в зависимости от архитектуры модели. Однако, если дать им информацию друг о друге во время компиляции, сеть удастся заметно ускорить. Выгружать программу для перекомпиляции, будь она движком инференса или СУБД — бессмысленно, поэтому компилировать надо во время работы, Just-In-Time. В предыдущей статье AI-дивизиона YADRO Илья Знаменский рассказывал про JIT на базе Xbyak. В продолжении темы, я расскажу про пет-проект векторной JIT-кодогенерации, который я веду, и покажу, как она может помогать в оптимизации.

Оптимизация, векторизация и JIT

Когда возникает задача оптимизации кода, а все мы хотим чтобы наше приложение или фреймворк не тормозили, есть несколько классических подходов. Если речь идет о вычислительно емких алгоритмах обработки значительных массивов данных, векторизация, то есть раскручивание циклов и задействование векторных инструкций SIMD , обрабатывающих пачку данных за раз, — это один из самых очевидных и мощных способов ускорения кода, работающего с однородными данными. Например, с данными компьютерного зрения, вычислительной физики, нейронных сетей, и даже, как ни странно, БД. Причем на мобильных устройствах альтернатива в виде портирования кода на GPU оказывается довольно дорогой, а иногда может даже проигрывать в скорости и энергоэффективности.

285572b62504c7d338147d6acf92c6ac.png

Почему это важно для «железа» заказчиков: в B2B-планшетах цена миллисекунды — это не только скорость интерфейса, но и время работы от батареи, нагрев, стабильность в длительных сессиях и предсказуемость на разных ревизиях устройств. Поэтому задачи оптимизации у нас приходят от полевых сценариев клиентов: распознавание/классификация на устройстве, пост-обработка, локальная аналитика – там, где облако недоступно или запрещено политиками безопасности.

Современные компиляторы могут автоматически векторизовать ваш код, но вы об этом не узнаете, пока не полезете в ассемблер, однако, несмотря на впечатляющее развитие возможностей автовекторизации, практика показывает, что иногда даже с несложными задачами они справиться не могут. Сложные же ядра автовекторизатор может либо не обработать, либо сделать это так, что из получившегося кода ручная оптимизация может выжать кратное ускорение. Например, программист может хитро изменить схему доступа к данным и порядок их обхода, заточив не только алгоритм, но и структуры данных специально под вектора. Поэтому работа с ассемблером и интринсиками более чем востребована.

Интринсики — это специализированные функции, которые почти в неизменном виде пролетают через пайплайн компилятора. Они плюс-минус соответствуют одной инструкции ассемблера. По сравнению с ассемблером интринсики добавляют очень ощутимый синтаксический сахар:

  • вы работаете не с ограниченным числом регистров, а с бесконечным числом переменных;

  • не нужно создавать ассемблерные вставки: код можно отлаживать вместе с остальным.

Конечно, это не избавляет оптимизатора от необходимости переосмыслить алгоритм в терминах векторных операций и даже периодически почитывать генерируемый ассемблерный код, но работа значительно упрощается. Интринсики — это главный инструмент человека, который занимается ручной векторизацией. Он органично и удобно описывает машинерию векторных расширений. В качестве простого примера посмотрим на упрощенный код функции сглаживания 3x3 на NEON, составляющей для каждого пикселя среднее из пикселей окрестности:

Код
cv::Mat src, dst;
// Специальный тип регистра с четырьмя float'ми
float32x4_t denominator = vdupq_n_f32(1./9.);
constexpr int NEON_32BIT_LANES = 4;
for(int y = 0; y < hd; y++)
{
    float* srcPrevLine = src.ptr<float>(y-1);
    float* srcCurLine  = src.ptr<float>(y);
    float* srcNextLine = src.ptr<float>(y+1);
    float* dstCurLine  = dst.ptr<float>(y);
    // Будем обрабатывать пиксели не по одному, а по четыре.
    for(int x = 0; x <= wd - NEON_32BIT_LANES; x+=NEON_32BIT_LANES)
    {
        float32x4_t sum = vld1q_f32(srcPrevLine + x - 1);
        // Эти функции со специфическими названиями и есть
        // интринсики, add - суммирование, ld - загрузка из
        // памяти, mul - умножение, st - выгрузка в память.
        // Немного практики - и все становится довольно просто.
        sum   = vaddq_f32(sum, vld1q_f32(srcPrevLine + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcPrevLine + x + 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x - 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x + 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x - 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x + 1));
        vst1q_f32(dstCurLine + x, vmulq_f32(sum, denominator));
    }
}

Давайте теперь поговорим про JIT и как JIT связан с оптимизацией. Классический подход к оптимизации — это использование обычного AOT-компилятора C++ с поддержкой интринсиков. Термин «JIT» наверняка хорошо знаком Java-программистам — он отсылает к Hotspot, стандартному JIT-компилятору, на ходу преобразующему вызываемые куски байт-кода в нативный код, который по ходу профилируется и, при необходимости, дооптимизируется, то есть компилируется с более агрессивными настройками. Поэтому часто люди, которые слышат «JIT-компилятор», представляют себе недоступный для управления программистом процесс автоматической компиляции байткода в машинный код.

Чтобы избежать путаницы, я буду пользоваться термином «JIT-кодогенерация», которая также осуществляется на лету, но находится под управлением программиста. Это совершенно другая практика, заключающаяся в написании программного кода в процессе работы программы с последующим запуском. В той же Java есть библиотека JIT-кодогенерации Janino, использующаяся, например, в Spark, для написания оптимального кода для фильтров конкретных запросов (насколько вообще может быть оптимальным Java-код).

В YADRO мы занимаемся оптимизацией под конкретные сценарии заказчиков. Часть таких сценариев живет на KVADRA_T – корпоративных планшетах: офлайн-инференс на устройстве, ограничения по энергии и термопакету, требования по безопасности и длительной поддержке. Поэтому JIT-кодогенерация и векторные оптимизации для нас — не абстракция, а способ выжать стабильную производительность в реальных B2B-нагрузках без зависимости от облака.

JIT-кодогенерация полезна в оптимизации — она позволяет на лету специализировать обобщенный код под текущий случай. Посмотрим на несколько примеров.

Пример 1. Удаление условных переходов

Самый простой пример — это удаление условных переходов, когда вы точно знаете, что какое-то условие не будет реализовано. AOT-компилятор с этим не справится, потому что ему неоткуда взять информацию о принципиальной нереализуемости ветвления.

Пример 2. Параметризованные ядра нейронных сетей

Пример, точнее передающий идею метапрограммирования, заложенную в JIT — это параметризованные ядра нейронных сетей, таких как свертки разных размеров, ресайзы или пулинги. Возьмем MaxPool, у которого несколько параметров: размер окна, stride и dilation. По классике сначала пишется общая реализация — медленная, но работающая при любых комбинациях параметров. Затем выбираются несколько наиболее часто употребляемых комбинаций ({3x3, stride 1x1}, {3x3, stride 2x2}, {5x5, stride 1x1},...) и переписываются оптимально. Но таких комбинаций много не напишешь, а их число очень быстро растет, увеличивая размер исходника и трудозатраты на написание и поддержку.

Наличие JIT же позволяет реализовать «метаоперацию MaxPool», которая создаст оптимальный код MaxPool для конкретных параметров, конкретного типа данных (float, uint8, …) и под конкретное железо (например, ARM v8.2).

В качестве PoC библиотеки Loops мы ускоряли ядра из экспериментального движка ficusNN, в частности MaxPool. MaxPool — сепарабельное ядро, то есть вместо O(N2) прохода по пикселям окна, его можно разбить на два O(N) прохода, сначала по вертикалям, потом по горизонталям. Однако, результат первого прохода нужно где-то хранить, обычно для этого используют память, а нам, благодаря JIT, удалось использовать под хранение промежуточного результата регистры. В виде обобщенной AOT-реализации сделать это в принципе невозможно: регистры хардкодятся в бинарное представление программы, перебирать их в цикле по номерам нельзя. Поэтому, либо вручную для отдельных комбинаций ядер, либо обобщенно с помощью JIT. В результате на типичных ядрах мы проигрывали ядрам ручной оптимизации небольшие проценты почти в пределах погрешности, а на экзотических, вроде 13 на 13, которые встречаются, например, в YOLO, получали ускорение в десятки раз, что позволяло ощутимо ускорять всю нейросеть.

Пример 3. Loop fusion

Лучший пример — это loop fusion, вынесенный в заголовок и давший название проекту. Два идущих подряд цикла с одним диапазоном и порядком, с независимыми данными или данными, зависимость которых можно уложить в итерацию. Циклы можно объединить в один, что позволит уменьшить накладные расходы на организацию циклов, количество операций чтения и записи, а если данные достаточно большие, то избежать лишней прокачки кэша.

Например, когда мы загружаем в память нейронную сеть и строим граф вычислений, мы можем проанализировать граф и понять, что какие-то операции можно и нужно объединить. В качестве примера идеально подойдет применение функций активации после какой-нибудь сложной по данным операции. Применение функции активации это поэлементная операция, поэтому может быть без труда подклеена к предыдущему слою, например к свертке или к полносвязному слою (матричному умножению).

Схема работы AOT- и JIT-компиляторов
Схема работы AOT- и JIT-компиляторов

Применимость JIT

В общем нам, как оптимизаторам, понятно, что JIT – это хорошо, векторный JIT — очень хорошо. А есть ли недостатки? В теории — нет. А вот чтобы решить, нужен ли вам на практике в вашем проекте JIT, нужно для себя ответить на несколько вопросов:

  • Скорость компиляции. Все-таки мы занимаемся оптимизацией, поэтому будет странно, если мы полминуты будем собирать код, чтобы ускорить его исполнение на долю секунды.

  • Добавление зависимостей. Компиляторы — это обычно большие проекты, интегрировать их в свой проект — значит усложнить сборку, число трудозатрат на поддержку этой сборки и заметно увеличить объем приложений.

  • Простота использования JIT. В идеале нам хотелось бы писать код как обычно мы пишем код с интринсиками, при этом каким-то образом указывая, какие переменные станут в собранном коде константным значением параметра, а какие будут именно переменными.

  • Кросс-платформенность. Векторизация — это низкоуровневая практика, сильно завязанная на архитектуру. Можно ли как-то добиться независимости наших кодогенераторов от типа машины?

Приглашаем в ИИ-команду YADRO, у нас есть задачи практически на любой вкус и для любого уровня:

→ AI Tech Lead (LLM)
→ Senior AQA Engineer (AI)
→ Product Owner AI Solutions

Мы, в составе небольшой команды из Вадима Писаревского и меня, постарались дать хорошие ответы на эти вопросы в нашем open source-проекте.

Мы задаем эти вопросы не в вакууме: в корпоративных планшетах и других персональных устройствах заказчиков любые «дорогие» решения быстро проявляются в эксплуатации — в автономности, температуре, SLA и стоимости владения. Поэтому R&D-задачи по JIT/векторизации у нас формулируются от требований клиентов: ускорить конкретный пайплайн, уложиться в энергобюджет, убрать зависимость от сети/облака, обеспечить воспроизводимость производительности на парке устройств.

Истоки проекта

Одним из источников вдохновения для нас стали universal intrinsics – технология из OpenCV. Фактически это abstraction layer. Такой подход позволяет писать код на векторных интринсиках независимо от платформы. Среди поддерживаемых платформ, например, Intel (AVX2/AVX512), ARM (Neon) и RISC-V (RVV). Благодаря технологии universal intrinsic большая часть векторизованных алгоритмов OpenCV написана разом под все поддерживаемые CPU-архитектуры. Это AOT-технология, поэтому ожидать способности избегать комбинаторного взрыва или ускорения кода с помощью слияния циклов тут не нужно.

Интересно, что установить полное соответствие между разными архитектурами невозможно: ряда инструкций нет в NEON, а других — в AVX2. Поэтому abstraction layer иногда приходится разворачивать отдельные его инструкции в небольшие последовательности нативных интринсиков конкретной платформы. Рассмотрим тот же пример с blur 3x3 на универсальных интринсиках:

Код
cv::Mat src, dst;
// Специальный тип регистра с четырьмя float'ми
float32x4_t denominator = vdupq_n_f32(1./9.);
constexpr int NEON_32BIT_LANES = 4;
for(int y = 0; y < hd; y++)
{
    float* srcPrevLine = src.ptr<float>(y-1);
    float* srcCurLine  = src.ptr<float>(y);
    float* srcNextLine = src.ptr<float>(y+1);
    float* dstCurLine  = dst.ptr<float>(y);
    // Будем обрабатывать пиксели не по одному, а по четыре.
    for(int x = 0; x <= wd - NEON_32BIT_LANES; x+=NEON_32BIT_LANES)
    {
        float32x4_t sum = vld1q_f32(srcPrevLine + x - 1);
        // Эти функции со специфическими названиями и есть
        // интринсики, add - суммирование, ld - загрузка из
        // памяти, mul - умножение, st - выгрузка в память.
        // Немного практики - и все становится довольно просто.
        sum   = vaddq_f32(sum, vld1q_f32(srcPrevLine + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcPrevLine + x + 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x - 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcCurLine  + x + 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x - 1));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x    ));
        sum   = vaddq_f32(sum, vld1q_f32(srcNextLine + x + 1));
        vst1q_f32(dstCurLine + x, vmulq_f32(sum, denominator));
    }
}

Второй и главный исток проекта, его прародитель — это Xbyak, библиотека JIT-кодогенерации от Сигэо Мицунари (Shigeo Mitsunari). Она используется в Intel OpenVINO для ускорения инференса нейронных сетей за счет компиляции параметризованных ядер (слоев нейронных сетей), специализации обобщенного кода и слияния циклов. Есть несколько версий Xbyak для:

Все версии Xbyak либо предоставляют возможность закодировать весь набор ассемблерных инструкций конкретной архитектуры, либо стремятся к этому. И это можно только приветствовать! Но именно поэтому нет особого смысла объединять эти проекты в один: точное следование архитектуре процессора ведет к необходимости писать под разные архитектуры разный пользовательский код.

Работа c Xbyak похожа на работу с ассемблером. Вы условно объявляете функцию (на самом деле создаете буфер в памяти) и начинаете вызывать «инструкции», то есть функции, имена которых повторяют названия ассемблерных инструкций. При вызове этих функций вместо исполнения инструкций происходит их добавление в конец буфера. Когда функция завершена, под нее специальным образом выделяется память, на нее можно получить указатель и запустить. Из-за полного соответствия набора функций ассемблерным командам трансляция происходит мгновенно. Это преимущество. Поскольку это JIT-кодогенерация, открываются все возможности ускорения с помощью специализации и слияния циклов.

Главная головная боль программиста на Xbyak – это распределение регистров. Любой реальный процессор имеет ограниченное количество регистров, которые с некоторой натяжкой можно спроецировать на понятие переменной. Например, у x86_64 — 16 регистров общего назначения и 16 векторных в AVX2/SSE или, если есть AVX512 — 32 векторных. Если отрезок кода использует больше переменных, их надо сохранять на стеке до момента использования (то есть имеет место так называемый register spilling).

В Xbyak процессами распределения регистров, загрузки, выгрузки переменных, отслеживания текущей локации — как в стеке, так и на регистре — управляет программист. Это тяжелая рутинная работа, знакомая всем, кто пишет на ассемблере. При использовании C/C++ или любого другого языка высокого уровня эти вопросы возьмет на себя компилятор, разгрузив голову человека. Но в Xbyak, несмотря на то что мета-функция пишется на C++, мы снова спускаемся на уровень ассемблера. В Loops мы тоже решаем этот вопрос. Это, пожалуй, его главное преимущество перед Xbyak.

Схожее преимущество — это работа с вызовами функций. Даже у одного процессора на разных операционных системах может использоваться разный calling convention – договоренность о том, как функции передаются аргументы. Часть из них размещается в регистрах, часть определенным образом укладывается в стек. Для правильного извлечения аргументов и подготовки фрейма стека у функций есть пролог и эпилог, которые в рамках работы с xbyak также должен писать программист. Поэтому одну и ту же Xbyak-функцию нужно писать по-разному для Windows и Linux. В Loops мы автоматизировали эту работу.

Пример с blur 3x3 выглядит громоздко, но зато мы в рантайме можем менять этот код в зависимости от наших потребностей:

Код
// Этот отрезок кода отличается от кода на NEON-интринсиках
// ровно одним: он запустится на Intel и Risc-V так же,
// как на Arm.
cv::Mat src, dst;
v_float32x4 denominator = v_setall_f32(1./9.);
constexpr int UI128_32BIT_LANES = 4;
for(int y = 0; y < hd; y++)
{
    float* srcPrevLine = src.ptr<float>(y-1);
    float* srcCurLine  = src.ptr<float>(y);
    float* srcNextLine = src.ptr<float>(y+1);
    float* dstCurLine  = dst.ptr<float>(y);
    for(int x = 0; x <= wd - UI128_32BIT_LANES; x+=UI128_32BIT_LANES)
    {
        v_float32x4 sum = v_load(srcPrevLine + x - 1);
        sum   = v_add(sum, v_load(srcPrevLine + x    ));
        sum   = v_add(sum, v_load(srcPrevLine + x + 1));
        sum   = v_add(sum, v_load(srcCurLine  + x - 1));
        sum   = v_add(sum, v_load(srcCurLine  + x    ));
        sum   = v_add(sum, v_load(srcCurLine  + x + 1));
        sum   = v_add(sum, v_load(srcNextLine + x - 1));
        sum   = v_add(sum, v_load(srcNextLine + x    ));
        sum   = v_add(sum, v_load(srcNextLine + x + 1));
        v_store(dstCurLine + x, v_mul(sum, denominator));
    }
}

Loops. Определение

Loops – это минималистичная скоростная кросс-платформенная C++-библиотека JIT-кодогенерации с распределением регистров, которая предназначена для написания векторного кода. То есть Loops — это JIT-компилятор, оформленный как библиотека, для удобной интеграции его в пользовательские проекты. Основные особенности — ниже.

Минимализм — не пытаясь повторить набор инструкций современных процессоров во всей их полноте, библиотека предоставляет все необходимые целочисленные скалярные инструкции для организации логики программы: базовых вычислений, циклов, условий, а также векторные инструкции, где в полной мере поддерживаются целочисленные операции и операции с плавающей запятой для всех основных типов данных, от 8-битных целых (int8_t/uint8_t) до 64-битных вещественных (double).

Кросс-платформенность — целочисленные скалярные инструкции реализованы для Intel/AMD x64, ARM aarch64 и RISC-V (тоже 64-бит). Векторные инструкции пока реализованы только для x64 (AVX2) и aarch64 (NEON), поддержка расширения RVV для RISC-V в среднесрочных планах. В отличие от Xbyak, ключевая кросс-платформенная особенность Loops — использование одного и того же имени для семантически одинаковых операций, независимо от платформы. Например, вместо mm256max_ps (интринсик для AVX2) или vmaxq_f32 (аналогичный интринсик для NEON) мы просто пишем loops::max, а тип операции (f32) будет автоматически выведен из типов аргументов.

Поддержка операционных систем:

  • Windows (x86_64),

  • Linux (x86_64, aarch64, riscv64),

  • MacOS (aarch64).

Из кроссплатформенности следует необходимость в платформо-независимом промежуточном представлении, о нем подробнее будет рассказано чуть ниже.

Скорость — Loops является очень быстрым многопроходным компилятором. Каждая функция компилируется независимо, и каждый проход имеет теоретическую сложность не больше чем O(N2) от числа команд N в функции, да и те в-среднем работают за O(N) или O(N*log(N)). На данный момент проект еще не профилировался, но предполагается, что можно дополнительно ускорить компиляцию за счет правильного подбора std-контейнеров, мелкой алгоритмической оптимизации и так далее. Loops сложно назвать оптимизирующим компилятором, предполагается, что он будет использоваться людьми с опытом векторизации с помощью интринсиков. Это очевидный и главный недостаток при сравнении его с большими компиляторными проектами вроде LLVM.

Распределение регистров — из-за желания добиться максимальной скорости компиляции вместо тяжеловесных алгоритмов раскраски графа используется усовершенствованный вариант простого алгоритма linear scan. Опыт показывает, что для наших задач этого вполне достаточно для получения качественного кода, особенно на современных платформах вроде ARM или RISC-V с большим количеством регистров: 32. Качество определяется количеством промежуточных сохранений регистров на стек с последующим восстановлением (так называемых «спиллов» / register spill). Чем их меньше, тем лучше. Важнее всего минимизировать количество «спиллов» во внутренних циклах, чего наша реализация linear scan и пытается добиться.

Простота сборки — хотя Loops, в отличие от Xbyak, не является суперлегковесным проектом, состоящим всего из нескольких заголовочных файлов, но довольно к этому близок — он состоит из нескольких заголовочных и исходных файлов на С++. Собирается с помощью CMake и не имеет внешних зависимостей.

Как упростить процесс отладки в CMake с помощью встроенного отладчика и профилировщика — читайте в статье «Как победить CMake: отладка CMake-скриптов».

Внутренняя машинерия

Разберемся с архитектурными решениями проекта.

Кросс-платформенность неизбежно требует введения IR. Исторически, первой архитектурой, которую мы поддержали в Loops, была ARM aarch64 с NEON, поэтому во многом IR повторяет их систему команд, в целом, довольно полную и удобную. Loops позволяет для каждой сгенерированной функции распечатать IR. Также можно распечатать в виде ассемблера получившийся из IR машинный код.

Исходя из минимализма, мы решили сделать структуры данных максимально простыми. IR-инструкции укладываются в простую планарную последовательность, без деревьев вроде AST. Это помешало бы написанию сложных оптимизирующих проходов, но для наших задач в самый раз — просто и быстро. Впрочем, на самом раннем этапе, при сборе кода в буфер, деревья все-таки используются, но это касается в основном интерфейса, а основная работа происходит над планарным представлением.

Идеальной простоты Xbyak, где псевдоинструкция сразу транслирует код и добавляет его в буфер, нам достигнуть не удастся, да и не хочется. Тут есть проходы по функциям как массивам IR-инструкций, уложенные в пайплайн, выполняющий компиляторную работу. Ниже мы разберем проходы по отдельности.

Схема пайплайна Loops
Схема пайплайна Loops

Интерфейс Loops имитирует привычные действия с кодом, как это делает Xbyak, но при этом содержит гораздо больше синтаксического сахара. Здесь вы не встретите команду mov — вместо нее нужно присваивать переменные и делать арифметические операции, что достигается благодаря агрессивной перегрузке операторов. Есть конструкции для цикла while и ветвлений. Одна из странностей — отсутствие оператора goto, что компенсируется другой странностью — операторами break(x) и continue(x), которые позволяют выходить не из самого вложенного цикла, а из нескольких вложений.

В течение работы проходы анализируют инструкции, опрашивая объект бэкэнда для выявления различных свойств той или иной инструкции. Например, самый первый проход после сбора буфера функции — это CP_IMMEDIATE_IMPLANTATION. Он определяет, можно ли вписать константный аргумент прямо в тело инструкции, или придется добавить отдельный mov. На Arm и RISC-V с их фиксированным размером инструкций такие вопросы возникают особенно часто. Для этого достаточно посмотреть, сколько места в бинарном представлении инструкции отведено под операнд, то есть рассмотреть энкодинг, который формируется в самом конце пайплайна и сильно зависит от целевой платформы. Проход работает с платформо-независимым IR, но при этом вынужден опираться на свойства инструкций нативной архитектуры, как бы подсматривать, что будет с этой IR-инструкцией в конце пайплайна. Такие опросы о свойствах инструкции распространены во всем пайплайне и обобщенно называются концепцией peeking. Она делает проходы универсальными, а всю необходимую информацию об инструкциях компактно собирает в одном месте: в энкодинге.

Список компиляторных проходов

Рассмотрим проходы по порядку. Префикс CP_ означает compiler pass.

  1. CP_COLLECTING — это метапроход, обозначающий состояние до начала основной обработки, в котором происходит сбор инструкций для функции, которую мы будем компилировать. Интерфейс при этом все же выполняет некоторую работу: например, разворачивает деревья выражений для присваиваний и использования в условиях циклов и ветвлений. На этом этапе циклы и ветвления помечаются специальными аннотационными инструкциями, которые позже исчезнут из кода. Например, для циклов есть три такие аннотации: annotation:whilecstart, annotation:whilecend и annotation:endwhile. Это компенсирует отсутствие древовидной структуры кода и фактически позволяет воссоздавать ее по этим аннотациям в тех проходах, где она нужна: анализе времени жизни и аллокации регистров.

  2. CP_IMMEDIATE_IMPLANTATION — уже упомянутый проход, распределяющий константные значения в телах инструкций, делая peeking энкодинга для определения ширины поля в битах.

  3. CP_ELIF_ELIMINATION — удаляет elif-инструкции, превращая их в систему вложенных if-else-endif.

  4. CP_<ARCHNAME>_BRA_SNIPPETS. BRA_SNIPPETS расшифровывается как Before Register Allocation Snippets и решает задачу, близкую к Universal Intrinsics в OpenCV: разворачивает инструкцию в последовательность инструкций, если на текущей архитектуре нет ее прямого аналога. <ARCHNAME> выделено потому, что на всех машинах имя отличается. В Loops есть система платформо-зависимых проходов, которые добавляются в двух точках: до аллокации регистров и после. Это задел под систему пользовательских проходов.

  5. CP_LIVENESS_ANALYSIS — один из двух наиболее алгоритмически сложных проходов. Он выполняет подготовительную работу для прохода распределения регистров и тесно с ним связан. На выходе у него не только и не столько преобразованный код, сколько карта времен жизни переменных. Кроме того, проход разбивает время жизни переменных на минимально возможные отрезки, что улучшает качество работы аллокатора регистров. Интересно, что в общем случае эта задача решается за O(N^4), однако отказ от произвольных goto позволяет снизить сложность до O(N^2), а в большинстве случаев — до O(M*N), где N — число инструкций, а M — число базовых блоков. Именно поэтому в Loops нет произвольных goto, но есть break(x) и continue(x). На этом этапе выполняется peeking информации о том, какие аргументы инструкции являются входными, а какие — выходными.

  6. CP_REGISTER_ALLOCATION — сердце Loops, собирающее все ниточки в одном месте. Здесь происходит переход от бесконечного числа скалярных и векторных переменных к конечному числу регистров, доступных на целевой машине. Если регистра под переменную не хватает, место под нее выделяется на стеке и добавляются инструкции spill и unspill. Вместо тяжеловесных алгоритмов раскраски графа используется простой, быстрый и, на практике, вполне эффективный алгоритм линейного сканирования. [Poletto, M. Linear scan register allocation / M. Poletto, V. Sarkar // ACM Transactions on Programming Languages and Systems (TOPLAS). —1999. — Т. 21, № 5. — С. 895—913.] Удивительно, но и этот, и предыдущий проход удалось сделать платформо-независимыми, ограничившись peeking энкодинга. Здесь же добавляются пролог и эпилог функции — они тесно связаны с набором использованных регистров.

  7. CP_CONTROLFLOW_TO_JUMPS — простой проход, который удаляет annotation-инструкции и заменяет их обычными командами условного и безусловного перехода.

  8. CP_<ARCHNAME>_ARA_SNIPPETS — это технические ARA-сниппеты (After Register Allocation), которые зависят не только от платформы, но и от ее регистров, поэтому выполняются после распределения регистров. Например, операция скалярного целочисленного деления на Intel требует, чтобы делимое и результат деления находились в регистре rax. Поэтому rax сохраняется на стеке, используется для деления, а затем восстанавливается. Одна из самых важных задач, которые решают ARA-сниппеты — вызов функций из сгенерированного кода.

  9. CP_IR_TO_ASSEMBLY — преобразование общего IR в ассемблерное представление. К этому этапу почти все инструкции переводятся простым отображением «команда в команду», «аргумент в аргумент». Иногда нужно поменять порядок аргументов или добавить дополнительные, но это несложная работа. Кроме того, на этом этапе появляется информация о размере кода, а значит о сдвигах, поэтому здесь переходы на метки превращаются в переходы на адреса, для вычисления этих сдвигов peeking запрашивает размеры команд.

  10. CP_ASSEMBLY_TO_HEX — перевод ассемблерного представления в бинарный вид и формирование тела функции в специально выделенном буфере. В основном это просто склеивание бинарных полей, но иногда бывают сложные случаи: например, с большими константами на Arm, где в ряде инструкций нужно обрабатывать как маленькие immediate, так и большие, из-за чего способ кодирования весьма нетривиальный. Энкодинг на Intel с его бесконечными слоями legacy самый громоздкий и запутанный. В то же время у минималистичного RISC-V есть всего четыре-пять заранее определенных форматов команд, работать с ним одно удовольствие.

Примеры использования Loops

Пример 1. Сложение

Для начала создадим какой-нибудь совсем простой пример и посмотрим, как он скомпилируется на разных машинах. Hello world на ассемблере писать не так-то просто, поэтому начнем с функции, которая складывает два входных числа:

#include <iostream>
#include <loops/loops.hpp>


int main()
{
    loops::Context CTX;          // Создаем контекст 
    USE_CONTEXT_(CTX);           // Обеспечиваем корректную работу макросов
    loops::IReg a, b;            // Создаем переменные-аргументы функции
    STARTFUNC_("sum", &a, &b)    // Объявляем новую функцию с именем и аргументами
    {                         
        loops::IReg c = a + b;   // Объявили переменную и присвоили ей результат сложения
        RETURN_(c);              // Макрос для возврата
    } 
    loops::Func sfunc = CTX.getFunc("sum"); // Достали по имени функцию из контекста
    std::cout << "=========IR LISTING=========\n"; 
    sfunc.printIR(std::cout);               // Печать IR(самый поздний проход)
    std::cout << "\n========ASM LISTING=========\n";
    sfunc.printAssembly(std::cout);         // Печать ассемблерного кода
    typedef int64_t (*sum_t)(int64_t a, int64_t b);
    sum_t func = (sum_t)sfunc.ptr();        // Компилируем и достаем указатель на функцию
    std::cout << "\n\n5 + 4 = " << func(5, 4) << "\n"; // Вызываем наш первый JIT!
    return 0;
}

Итак, по пунктам:

  1. Сначала нужно создать контекст (loops::Context). Это контейнер функций, с помощью него они создаются и удаляются, доступ к функциям по именам.

  2. Если вы собираетесь что-то писать в буфер текущей функции, то нужно инициализировать макросы с помощью USE_CONTEXT_. Если только брать уже написанные функции, печатать их код или запускать, USE_CONTEXT_ не нужен.

  3. Благодаря USE_CONTEXT_ хорошо сработает следующий макрос — STARTFUNC_, который «объявляет» функцию. В фигурных скобках внутри можно писать весь нужный вам код. Само собой, описываемые вами действия — не выполняются, вы только добавляете в буфер их последовательность, прямо во время выполнения программы решая, какую последовательность действий туда записать.

  4. Loops::IReg — абстракция для регистра общего назначения, в более широком смысле — это переменная. Это всегда 64-битное знаковое целое, но вы можете загружать туда и выгружать оттуда типы меньших размеров — поддержаны соответствующие инструкции.

  5. Loops::IReg должен объявляться только внутри блока объявленной функции, иначе будут ошибки. Единственное исключение — это объявление аргументов функций. Они должны объявляться по умолчанию и подаваться в макрос STARTFUNC_ для обозначения аргументов генерируемой нами функции.

  6. Внутри функции вы можете пользоваться loops::IReg, как обычной переменной: складывать, присваивать и передавать в функции. Она будет вести себя примерно так же, как и обычная переменная. Однако, на первых порах не рекомендую объявлять переменные по умолчанию, потому что им неоткуда будет взять информацию о контексте.

  7. Loops::Func описывает функцию, из нее можно получить указатель void* на скомпилированный код. Также позволяет распечатать функцию, причем на разных стадиях обработки: можно указать имя прохода, и printIR выведет код после применения этого прохода.

  8. Перед использованием этот указатель надо преобразовать в соответствии с сигнатурой, и можно выполнять свежескомпилированный код.

Что выведет эта программа на Windows+x86_64:

=========IR LISTING=========
sum(i0, i1)
     0 : add i1, i1, i2
     1 : mov i0, i1
     2 : ret

========ASM LISTING=========
sum(i0, i1)
     0 : add rcx, rdx ; 48 01 d1
     1 : mov rax, rcx ; 48 89 c8
     2 : ret          ; c3


5 + 4 = 9

Отмечу, что ASM listing будет выглядеть иначе, даже если просто перейти с одной операционной системы на другую, в нашем случае на Linux. Это связано с тем, что в Windows и Linux используются разные соглашения о том, через какие регистры передаются аргументы:

========ASM LISTING=========
sum(i0, i1)
     0 : add rdi, rsi ; 48 01 f7  
     1 : mov rax, rdi ; 48 89 f8  
     2 : ret          ; c3

Тот же листинг на Arm:

========ASM LISTING=========
sum(i0, i1)
     0 : add x0, x0, x1 ; 00 00 01 8b  
     1 : ret x30        ; c0 03 5f d6

И на RISC-V:

========ASM LISTING=========
sum(i0, i1)
     0 : add a0, a0, a1 ; 33 05 b5 00  
     1 : ret            ; 67 80 00 00

Пример 2. Сортировка

Первый пример был довольно простым, поэтому напишем программу посложнее, чтобы посмотреть, как организуются циклы и ветвления.

Далее я привожу примеры на Arm, чтобы не тратить время на объяснения архитектурных различий.

Рассмотрим простую сортировку целых 32-битных знаковых чисел:

#include <iostream>
#include <loops/loops.hpp>


int main()
{
    loops::Context CTX;
    USE_CONTEXT_(CTX);
    loops::IReg ptr32, size;
    STARTFUNC_("sort", &ptr32, &size)
    {
        loops::IReg curnum = CONST_(0);   // Начальное значение переменной задается с помощью макроса CONST_
        WHILE_(curnum < size - 1)         // Цикл, как цикл. Но генерирует код, а не гоняет итерации.
        {
            loops::IReg minnum = curnum;  
            loops::IReg searchnum = curnum + 1;
            WHILE_(searchnum < size - 1)
            {
                // Обращение к элементам массива - задача чуть более сложная, чем в Си. И обратите
                // внимание, что offset домножается на размер int32_t. Без типизации 
                // адресная арифметика становится сложнее.
                loops::IReg minval    = loops::load_<int32_t>(ptr32, minnum * sizeof(int32_t));
                loops::IReg searchval = loops::load_<int32_t>(ptr32, searchnum * sizeof(int32_t));
                IF_(searchval < minval)
                    minnum = searchnum;
                searchnum = searchnum + 1;
            }
            loops::IReg curval    = loops::load_<int32_t>(ptr32, curnum * sizeof(int32_t));
            loops::IReg minval    = loops::load_<int32_t>(ptr32, minnum * sizeof(int32_t));
            loops::store_<int32_t>(ptr32, curnum * sizeof(int32_t), minval);
            loops::store_<int32_t>(ptr32, minnum * sizeof(int32_t), curval);
            curnum = curnum + 1;
        }
    } 
    loops::Func sfunc = CTX.getFunc("sort");
    std::cout << "=========IR LISTING=========\n"; 
    sfunc.printIR(std::cout);
    std::cout << "\n========ASM LISTING=========\n";
    sfunc.printAssembly(std::cout);
    typedef void (*sort_t)(int32_t* ptr32, int64_t size);
    sort_t func = (sort_t)sfunc.ptr();
    std::vector<int32_t> toSort = {5, 2, 15, -4, 10};
    func(toSort.data(), (int64_t)toSort.size());
    std::cout << "\n========SORTED VALUES=========\n";
    for(int i = 0; i < (int)toSort.size(); i++)
        std::cout << toSort[i] << std::endl;
    return 0;
}
Вывод
=========IR LISTING=========
sort(i0, i1)
     0 : mov              i2, 0
     1 : __loops_label_0:
     2 : sub              i3, i1, 1
     3 : cmp              i2, i3
     4 : jmp_ge           __loops_label_2
     5 : mov              i3, i2
     6 : add              i4, i2, 1
     7 : __loops_label_3:
     8 : sub              i5, i1, 1
     9 : cmp              i4, i5
    10 : jmp_ge           __loops_label_5
    11 : mov              i5, 4
    12 : mul              i5, i3, i5
    13 : load.i32         i5, i0, i5
    14 : mov              i6, 4
    15 : mul              i6, i4, i6
    16 : load.i32         i6, i0, i6
    17 : cmp              i6, i5
    18 : jmp_ge           __loops_label_7
    19 : mov              i3, i4
    20 : __loops_label_7:
    21 : add              i4, i4, 1
    22 : jmp              3
    23 : __loops_label_5:
    24 : mov              i4, 4
    25 : mul              i4, i2, i4
    26 : load.i32         i4, i0, i4
    27 : mov              i5, 4
    28 : mul              i5, i3, i5
    29 : load.i32         i5, i0, i5
    30 : mov              i6, 4
    31 : mul              i6, i2, i6
    32 : store.i32        i0, i6, i5
    33 : mov              i5, 4
    34 : mul              i3, i3, i5
    35 : store.i32        i0, i3, i4
    36 : add              i2, i2, 1
    37 : jmp              0
    38 : __loops_label_2:
    39 : ret

========ASM LISTING=========
sort(i0, i1)
     0 : eor   x2, x2, x2       ; 42 00 02 ca
     1 :       __loops_label_0: ;
     2 : sub   x3, x1, #0x01    ; 23 04 00 d1
     3 : cmp   x2, x3           ; 5f 00 03 eb
     4 : b.ge  __loops_label_2  ; ea 03 00 54
     5 : mov   x3, x2           ; e3 03 02 aa
     6 : add   x4, x2, #0x01    ; 44 04 00 91
     7 :       __loops_label_3: ;
     8 : sub   x5, x1, #0x01    ; 25 04 00 d1
     9 : cmp   x4, x5           ; 9f 00 05 eb
    10 : b.ge  __loops_label_5  ; 8a 01 00 54
    11 : mov   x5, #0x04        ; 85 00 80 d2
    12 : mul   x5, x3, x5       ; 65 7c 05 9b
    13 : ldrsw x5, [x0, x5]     ; 05 68 a5 b8
    14 : mov   x6, #0x04        ; 86 00 80 d2
    15 : mul   x6, x4, x6       ; 86 7c 06 9b
    16 : ldrsw x6, [x0, x6]     ; 06 68 a6 b8
    17 : cmp   x6, x5           ; df 00 05 eb
    18 : b.ge  __loops_label_7  ; 4a 00 00 54
    19 : mov   x3, x4           ; e3 03 04 aa
    20 :       __loops_label_7: ;
    21 : add   x4, x4, #0x01    ; 84 04 00 91
    22 : b     __loops_label_3  ; f3 ff ff 17
    23 :       __loops_label_5: ;
    24 : mov   x4, #0x04        ; 84 00 80 d2
    25 : mul   x4, x2, x4       ; 44 7c 04 9b
    26 : ldrsw x4, [x0, x4]     ; 04 68 a4 b8
    27 : mov   x5, #0x04        ; 85 00 80 d2
    28 : mul   x5, x3, x5       ; 65 7c 05 9b
    29 : ldrsw x5, [x0, x5]     ; 05 68 a5 b8
    30 : mov   x6, #0x04        ; 86 00 80 d2
    31 : mul   x6, x2, x6       ; 46 7c 06 9b
    32 : str   w5, [x0, x6]     ; 05 68 26 b8
    33 : mov   x5, #0x04        ; 85 00 80 d2
    34 : mul   x3, x3, x5       ; 63 7c 05 9b
    35 : str   w4, [x0, x3]     ; 04 68 23 b8
    36 : add   x2, x2, #0x01    ; 42 04 00 91
    37 : b     __loops_label_0  ; e0 ff ff 17
    38 :       __loops_label_2: ;
    39 : ret   x30              ; c0 03 5f d6

========SORTED VALUES=========
-4
2
5
15
10

Посмотрим, на что в этом примере стоит обратить внимание.

В коде можно встретить макрос CONST_ при объявлении переменных. Дело в том, что Loops должен помещать операции в буфер программы, а для этого ему нужно знать в какой именно буфер. Эта информация обычно хранится в операндах — уже объявленных IReg. Это сделано ради синтаксического сахара и лаконичности выражений. Однако только конструирующемуся IReg взять эту информацию неоткуда. CONST_ порождает аргумент уже с ней, подключая IReg к буферу. Также можно объявить переменную без инициализации, с помощью макроса DEF_().

В Loops переменные описывают обобщенные регистры, что характерно для низкоуровневого кода. Память при этом рассматривается как внешняя среда, взаимодействие с которой происходит через специальные операции ввода/вывода (load/store). Также нужно обратить внимание на особенности адресной арифметики. Типизация указателей не используется, поэтому указатели и их смещения измеряются в байтах. Возможно, позже мы это улучшим.

Пример 3. Векторы и вызовы

Концептуально loops выстроен вокруг SIMD-регистров. Давайте поработаем с ними и напишем простой одномерный maxpool: в каждом элементе целевого массива — максимальное значение из трех подряд идущих элементов оригинала.

Код упрощенный, размеры входных массивов нужно проверять снаружи. Чтобы показать еще что-нибудь интересное, добавим отладочный вызов, который будет выводить некоторые значения:

// Аргументы вызываемых функций - только int64_t
// Возврат - либо void, либо int64_t
void debug_print(int64_t a)
{
    float* a_f32 = (float*)(void*)(&a);
    printf("Loops debug: %f\n", *a_f32);
}


//...
STARTFUNC_("maxpool1d_3", &ptrin_f32, &ptrout_f32, &sizeout)
{ 
    loops::IReg curpos = CONST_(0);
    WHILE_(curpos < sizeout)
    {
        // Векторные регистры типизированы, благодаря чему под арифметические и иные действия
        // автоматически подбираются корректные инструкции
        loops::VReg<float> in0 = loops::loadvec<float>(ptrin_f32, curpos       * sizeof(float));
        loops::VReg<float> in1 = loops::loadvec<float>(ptrin_f32, (curpos + 1) * sizeof(float));
        loops::VReg<float> in2 = loops::loadvec<float>(ptrin_f32, (curpos + 2) * sizeof(float));
        loops::VReg<float> out = loops::max(loops::max(in0, in1), in2);
        VOID_CALL_(debug_print, loops::getlane(out, 1)); // Вызов функции с одним аргументом
        loops::storevec(ptrout_f32, curpos * sizeof(float), out);
        curpos = curpos + CTX.vlanes<float>();// Количество lane'ов в векторе на разных машинах разное
    }
}
Полученный код
=========IR LISTING=========
maxpool1d_3(i0, i1, i2)
     0 : sub              i31, i31, 416
     1 : arm_stp          i31, 400, i29, i30
     2 : mov              i29, i31
     3 : mov              i3, 0
     4 : __loops_label_0:
     5 : cmp              i3, i2
     6 : jmp_ge           __loops_label_2
     7 : mov              i4, 4
     8 : mul              i4, i3, i4
     9 : vld.fp32         v0, i0, i4
    10 : add              i4, i3, 1
    11 : mov              i5, 4
    12 : mul              i4, i4, i5
    13 : vld.fp32         v1, i0, i4
    14 : add              i4, i3, 2
    15 : mov              i5, 4
    16 : mul              i4, i4, i5
    17 : vld.fp32         v2, i0, i4
    18 : max.fp32         v0, v0, v1
    19 : max.fp32         v0, v0, v2
    20 : getlane.fp32     i4, v0, 1
    21 : mov              i5, 25416
    22 : arm_movk         i5, 47206, 16
    23 : arm_movk         i5, 43690, 32
    24 : arm_stp          i31, 0, i0, i1
    25 : arm_stp          i31, 16, i2, i3
    26 : arm_stp          i31, 32, i4, i5
    27 : arm_stp          i31, 48, i6, i7
    28 : arm_stp          i31, 64, i8, i9
    29 : arm_stp          i31, 80, i10, i11
    30 : arm_stp          i31, 96, i12, i13
    31 : arm_stp          i31, 112, i14, i15
    32 : arm_stp          i31, 128, i16, i17
    33 : mov              i0, i4
    34 : add              i10, i31, 144
    35 : vst_lane.u64     i10, v0
    36 : vst_lane.u64     i10, v4
    37 : vst_lane.u64     i10, v8
    38 : vst_lane.u64     i10, v12
    39 : call_noret       [i5]()
    40 : add              i10, i31, 144
    41 : vld_lane.u64     v0, i10
    42 : vld_lane.u64     v4, i10
    43 : vld_lane.u64     v8, i10
    44 : vld_lane.u64     v12, i10
    45 : arm_ldp          i0, i1, i31, 0
    46 : arm_ldp          i2, i3, i31, 16
    47 : arm_ldp          i4, i5, i31, 32
    48 : arm_ldp          i6, i7, i31, 48
    49 : arm_ldp          i8, i9, i31, 64
    50 : arm_ldp          i10, i11, i31, 80
    51 : arm_ldp          i12, i13, i31, 96
    52 : arm_ldp          i14, i15, i31, 112
    53 : arm_ldp          i16, i17, i31, 128
    54 : mov              i4, 4
    55 : mul              i4, i3, i4
    56 : vst.fp32         i1, i4, v0
    57 : add              i3, i3, 4
    58 : jmp              0
    59 : __loops_label_2:
    60 : arm_ldp          i29, i30, i31, 400
    61 : add              i31, i31, 416
    62 : ret

========ASM LISTING=========
maxpool1d_3(i0, i1, i2)
     0 : sub  sp, sp, #0x1a0                                 ; ff 83 06 d1
     1 : stp  x29, x30, [sp, #0x190]                         ; fd 7b 19 a9
     2 : mov  x29, sp                                        ; fd 03 1f aa
     3 : eor  x3, x3, x3                                     ; 63 00 03 ca
     4 :      __loops_label_0:                               ;
     5 : cmp  x3, x2                                         ; 7f 00 02 eb
     6 : b.ge __loops_label_2                                ; aa 06 00 54
     7 : mov  x4, #0x04                                      ; 84 00 80 d2
     8 : mul  x4, x3, x4                                     ; 64 7c 04 9b
     9 : ldr  q0, [x0, x4]                                   ; 00 68 e4 3c
    10 : add  x4, x3, #0x01                                  ; 64 04 00 91
    11 : mov  x5, #0x04                                      ; 85 00 80 d2
    12 : mul  x4, x4, x5                                     ; 84 7c 05 9b
    13 : ldr  q1, [x0, x4]                                   ; 01 68 e4 3c
    14 : add  x4, x3, #0x02                                  ; 64 08 00 91
    15 : mov  x5, #0x04                                      ; 85 00 80 d2
    16 : mul  x4, x4, x5                                     ; 84 7c 05 9b
    17 : ldr  q2, [x0, x4]                                   ; 02 68 e4 3c
    18 : fmax v0.4s, v0.4s, v1.4s                            ; 00 f4 21 4e
    19 : fmax v0.4s, v0.4s, v2.4s                            ; 00 f4 22 4e
    20 : umov w4, v0.s[1]                                    ; 04 3c 0c 0e
    21 : mov  x5, #0x6348                                    ; 05 69 8c d2
    22 : movk x5, #0xb866, lsl #16                           ; c5 0c b7 f2
    23 : movk x5, #0xaaaa, lsl #32                           ; 45 55 d5 f2
    24 : stp  x0, x1, [sp, #0]                               ; e0 07 00 a9
    25 : stp  x2, x3, [sp, #0x10]                            ; e2 0f 01 a9
    26 : stp  x4, x5, [sp, #0x20]                            ; e4 17 02 a9
    27 : stp  x6, x7, [sp, #0x30]                            ; e6 1f 03 a9
    28 : stp  x8, x9, [sp, #0x40]                            ; e8 27 04 a9
    29 : stp  x10, x11, [sp, #0x50]                          ; ea 2f 05 a9
    30 : stp  x12, x13, [sp, #0x60]                          ; ec 37 06 a9
    31 : stp  x14, x15, [sp, #0x70]                          ; ee 3f 07 a9
    32 : stp  x16, x17, [sp, #0x80]                          ; f0 47 08 a9
    33 : mov  x0, x4                                         ; e0 03 04 aa
    34 : add  x10, sp, #0x90                                 ; ea 43 02 91
    35 : st1  {v0.2d, v1.2d, v2.2d, v3.2d}, [x10], #0x40     ; 40 2d 9f 4c
    36 : st1  {v4.2d, v5.2d, v6.2d, v7.2d}, [x10], #0x40     ; 44 2d 9f 4c
    37 : st1  {v8.2d, v9.2d, v10.2d, v11.2d}, [x10], #0x40   ; 48 2d 9f 4c
    38 : st1  {v12.2d, v13.2d, v14.2d, v15.2d}, [x10], #0x40 ; 4c 2d 9f 4c
    39 : blr  x5                                             ; a0 00 3f d6
    40 : add  x10, sp, #0x90                                 ; ea 43 02 91
    41 : ld1  {v0.2d, v1.2d, v2.2d, v3.2d}, [x10], #0x40     ; 40 2d df 4c
    42 : ld1  {v4.2d, v5.2d, v6.2d, v7.2d}, [x10], #0x40     ; 44 2d df 4c
    43 : ld1  {v8.2d, v9.2d, v10.2d, v11.2d}, [x10], #0x40   ; 48 2d df 4c
    44 : ld1  {v12.2d, v13.2d, v14.2d, v15.2d}, [x10], #0x40 ; 4c 2d df 4c
    45 : ldp  x0, x1, [sp, #0]                               ; e0 07 40 a9
    46 : ldp  x2, x3, [sp, #0x10]                            ; e2 0f 41 a9
    47 : ldp  x4, x5, [sp, #0x20]                            ; e4 17 42 a9
    48 : ldp  x6, x7, [sp, #0x30]                            ; e6 1f 43 a9
    49 : ldp  x8, x9, [sp, #0x40]                            ; e8 27 44 a9
    50 : ldp  x10, x11, [sp, #0x50]                          ; ea 2f 45 a9
    51 : ldp  x12, x13, [sp, #0x60]                          ; ec 37 46 a9
    52 : ldp  x14, x15, [sp, #0x70]                          ; ee 3f 47 a9
    53 : ldp  x16, x17, [sp, #0x80]                          ; f0 47 48 a9
    54 : mov  x4, #0x04                                      ; 84 00 80 d2
    55 : mul  x4, x3, x4                                     ; 64 7c 04 9b
    56 : str  q0, [x1, x4]                                   ; 20 68 a4 3c
    57 : add  x3, x3, #0x04                                  ; 63 10 00 91
    58 : b    __loops_label_0                                ; cb ff ff 17
    59 :      __loops_label_2:                               ;
    60 : ldp  x29, x30, [sp, #0x190]                         ; fd 7b 59 a9
    61 : add  sp, sp, #0x1a0                                 ; ff 83 06 91
    62 : ret  x30                                            ; c0 03 5f d6

Код довольно наглядный, все же добавлю пару пояснений

CTX.vlanes<float> возвращает количество элементов данного типа, которые умещаются в векторном регистре. Если мы говорим про float, то на Intel (AVX2) их будет восемь, а на Arm (NEON) — четыре. Понятно, что если мы ведем обработку векторами, то «шагать» по массиву надо именно на такое количество.

Отладка JIT-кода — задача не из простых. В лучшем случае приходится «ползать» по ассемблеру. Даже простая отладочная печать не всегда доступна — нельзя просто вставить вызов printf в код.

С появлением поддержки вызовов функций ситуация немного упростилась. Сигнатура вызываемых функций сильно ограничена. Аргументы могут быть только типа int64_t. Их количество зависит от платформы: на Intel — до четырех аргументов на Windows и до шести на Linux, на ARM и RISC-V — до восьми. То есть максимальное количество аргументов совпадает с количеством аргументов, которые передаются через регистры согласно соглашению о вызове функций (calling convention). В коде можно увидеть много инструкций spill. Это префикс и постфикс вызова. Другими словами, вызовы хоть и есть, реализованы не очень хорошо и пока стоит их избегать, по крайней мере в нагруженных участках кода.

Пример 4. Параметрический поэлементный полином

Приведем, наконец, пример, с которым может справиться только JIT — параметрическая компиляция. Loops очень хорошо подходит для поэлементных или попиксельных операций, поэтому рассмотрим функцию, которая поэлементно высчитывает полином. Параметрами будут степень и коэффициенты этого полинома. Не будем все это сильно оптимизировать, напишем проще и посмотрим код, который получается в зависимости от входных данных:

loops::Func create_polynomic_function(loops::Context& CTX, const std::vector<float> ratios)
{
    USE_CONTEXT_(CTX);
    loops::IReg ptrin_f32, ptrout_f32, sizeout;
    STARTFUNC_("poly", &ptrin_f32, &ptrout_f32, &sizeout)
    {
        loops::IReg curpos = CONST_(0);
        std::vector<loops::VReg<float> > ratiosV(ratios.size()); 
        for(int rNum = 0; rNum < ratios.size(); rNum++)
            if(rNum == 0 || (ratios[rNum] != 0.f && ratios[rNum] != 1.f))
            {// Отложенная инициализация регистра необходима при работе с массивом VReg.
                ratiosV[rNum].copyidx(VCONST_(float, ratios[rNum]));
            }
        WHILE_(curpos < sizeout)
        {
            loops::VReg<float> result = ratiosV[0];
            loops::VReg<float> x = loops::loadvec<float>(ptrin_f32, curpos * sizeof(float));
            // res = ratios[0] + ratios[1] * x * ratios[2] * x^2 + ...;
            loops::VReg<float> power = x; 
            // Цикл, добавляющий инструкции в зависимости от числа входных коэффициентов
            for(size_t pNum = 1; pNum < ratiosV.size(); pNum++) 
            {// Нулевые компоненты не добавляем, если коэффициент единица - плюсуем степень без умножения.
                if(ratios[pNum] == 1.f)              // Главное не путать if'ы и IF_'ы
                    result += power;
                else if(ratios[pNum] != 0.f)
                    result += power * ratiosV[pNum];
                if(pNum + 1 < ratiosV.size())
                    power *= x;
            }
            loops::storevec(ptrout_f32, curpos * sizeof(float), result);
            curpos = curpos + CTX.vlanes<float>();
        }
    }
    return CTX.getFunc("poly");
}


int main()
{
    loops::Context CTX;
    USE_CONTEXT_(CTX);
    loops::IReg ptrin_f32, sizeout, ptrout_f32;
    loops::Func sfunc = create_polynomic_function(CTX, {1});
    std::cout << "=========IR LISTING=========\n"; 
    // Мы можем указать проход после которого мы хотим напечатать код
    sfunc.printIR(std::cout, 3, "CP_COLLECTING");
    std::cout << "\n========ASM LISTING=========\n";
    sfunc.printAssembly(std::cout);
    return 0;
}

Посмотрим на код в том виде, в котором он появляется в буфере до всех проходов. Для этого достаточно в функцию printIR передать название нужного пасса, в данном случае “CP_COLLECTING”.

Для вырожденного полинома({1}):

poly(i0, i1, i2)
     0 : mov                    i3, 0
     1 : mov                    v0, 1065353216
     2 : annotation:whilecstart 0
     3 : cmp                    i3, i2
     4 : jmp_ge                 __loops_label_2
     5 : annotation:whilecend
     6 : mov                    v1, v0
     7 : mul                    i4, i3, 4
     8 : vld.fp32               v2, i0, i4
     9 : mov                    v3, v2
    10 : mul                    i5, i3, 4
    11 : vst.fp32               i1, i5, v1
    12 : add                    i3, i3, 8
    13 : annotation:endwhile    0, 2

Для квадратичного случая {1,2,1} :

poly(i0, i1, i2)
     0 : mov                    i3, 0
     1 : mov                    v0, 1065353216  
     2 : mov                    v1, 1073741824  
     3 : annotation:whilecstart 0
     4 : cmp                    i3, i2
     5 : jmp_ge                 __loops_label_2 
     6 : annotation:whilecend
     7 : mov                    v2, v0
     8 : mul                    i4, i3, 4       
     9 : vld.fp32               v3, i0, i4      
    10 : mov                    v4, v3
    11 : mul.fp32               v5, v4, v1      
    12 : add.fp32               v2, v2, v5
    13 : mul.fp32               v4, v4, v3
    14 : add.fp32               v2, v2, v4
    15 : mul                    i5, i3, 4
    16 : vst.fp32               i1, i5, v2
    17 : add                    i3, i3, 8
    18 : annotation:endwhile    0, 2

Для кубического случая {0.5, 0, 0, 2}:

poly(i0, i1, i2)
     0 : mov                    i3, 0
     1 : mov                    v0, 1056964608
     2 : mov                    v1, 1073741824
     3 : annotation:whilecstart 0
     4 : cmp                    i3, i2
     5 : jmp_ge                 __loops_label_2
     6 : annotation:whilecend
     7 : mov                    v2, v0
     8 : mul                    i4, i3, 4
     9 : vld.fp32               v3, i0, i4
    10 : mov                    v4, v3
    11 : mul.fp32               v4, v4, v3
    12 : mul.fp32               v4, v4, v3
    13 : mul.fp32               v5, v4, v1
    14 : add.fp32               v2, v2, v5
    15 : mul                    i5, i3, 4
    16 : vst.fp32               i1, i5, v2
    17 : add                    i3, i3, 8
    18 : annotation:endwhile    0, 2

Если не считать граничных случаев, которые можно аккуратно обработать в генераторе, получается оптимальный код.

Посмотрим, на что в этом примере стоит обратить внимание.

Отложенная инициализация регистров — важный момент. Думаю, уже понятно, что у синтаксического сахара, который позволяет маскировать добавление действий в буфер под сами эти действия, есть своя цена. В частности — сложности с инициализацией.

Золотое правило работы с Loops-регистрами — они должны инициализироваться сразу. Однако это не всегда возможно. Например, если нужно работать с целым контейнером регистров, то инициализацию приходится откладывать. В таких случаях на помощь приходит функция copyidx, которая позволяет создать несколько ссылок на один регистр, то есть скопировать его индекс.

Цикл, который увеличивает регистр с результатом, показывает главный принцип: в генерирующем коде мы видим понятный цикл, а в сгенерированном — фиксированную последовательность действий, без итераций. Когда код становится сложным, можно начать путать циклы генератора и циклы, которые мы создаем. Генерирующие и генерируемые управляющие конструкции путаются, это довольно странное ощущение, но со временем к этому привыкаешь.

Коэффициенты мы перевели в векторный вид заранее, предполагая, что полином будет не слишком большой и нам хватит векторных регистров. Часть регистров теперь зарезервирована под хранение коэффициентов.

Под катом — Neon-код последнего случая {0.5, 0, 0, 2}:

Код
========ASM LISTING=========
poly(i0, i1, i2)
     0 : eor  x3, x3, x3           ; 63 00 03 ca
     1 : eor  x4, x4, x4           ; 84 00 04 ca
     2 : movk x4, #0x3f00, lsl #16 ; 04 e0 a7 f2
     3 : dup  v0.4s, w4            ; 80 0c 04 4e
     4 : eor  x4, x4, x4           ; 84 00 04 ca
     5 : movk x4, #0x4000, lsl #16 ; 04 00 a8 f2
     6 : dup  v1.4s, w4            ; 81 0c 04 4e
     7 :      __loops_label_0:     ;
     8 : cmp  x3, x2               ; 7f 00 02 eb
     9 : b.ge __loops_label_2      ; ea 01 00 54
    10 : mov  v2.4s, v0.4s         ; 02 1c a0 4e
    11 : mov  x4, #0x04            ; 84 00 80 d2
    12 : mul  x4, x3, x4           ; 64 7c 04 9b
    13 : ldr  q3, [x0, x4]         ; 03 68 e4 3c
    14 : mov  v4.4s, v3.4s         ; 64 1c a3 4e
    15 : fmul v4.4s, v4.4s, v3.4s  ; 84 dc 23 6e
    16 : fmul v3.4s, v4.4s, v3.4s  ; 83 dc 23 6e
    17 : fmul v3.4s, v3.4s, v1.4s  ; 63 dc 21 6e
    18 : fadd v2.4s, v2.4s, v3.4s  ; 42 d4 23 4e
    19 : mov  x4, #0x04            ; 84 00 80 d2
    20 : mul  x4, x3, x4           ; 64 7c 04 9b
    21 : str  q2, [x1, x4]         ; 22 68 a4 3c
    22 : add  x3, x3, #0x04        ; 63 10 00 91
    23 : b    __loops_label_0      ; f1 ff ff 17
    24 :      __loops_label_2:     ;
    25 : ret  x30                  ; c0 03 5f d6

Замеры

Приведу небольшое сравнение скоростей обобщенного AOT-кода с кодом, сгенерированным в Loops. В качестве PoC мы написали два оптимальных типичных ядра: depthwise convolution и maxpool и посмотрели, как они будут работать отдельно и в составе сети. Заметим, что ускорение достигается за счет параметрической компиляции, а не других техник, вроде loops fusion. Конфигурация для замера: Apple M1, 8 Gb, Debian, 4 потока. В YOLO есть большие и очень редкие maxpool, в efficientnet есть depthwise convolution. Замеряемая статистическая характеристика: минимум времени в 100 повторениях.

Case

AOT

JIT

Speedup

Maxpool 13x13(fp16)

13.9 ms

0.39 ms

35.64x

YOLOv4(fp16)

173.33 ms

162.2 ms

1.07x

Depthwise convolution 5x5 (fp32)

3.48 ms

1.97 ms

1.77x

efficientnet-lite4-11(fp32)

12.61 ms

11.07 ms

1.14x

Сравнение с аналогами

Для полноты картины нужно сравнить Loops c похожими проектами и обозначить уникальную нишу нашего проекта.

Про Xbyak мы написали уже достаточно.

В любом разговоре о компиляторах сложно обойти стороной LLVM — пожалуй, самый главный компиляторный проект на сегодня. Он умеет все, включая JIT-компиляцию. Проблема в его размере, сложности использования, скорости компиляции и в том, что поддерживать зависимость от него не так-то просто. О one-header простоте в случае с LLVM, конечно, речи не идет.

Также упомяну очень интересный проект Владимира Макарова, MIR. Насколько я понимаю, на данный момент проект мигрирует в сторону полноценного легковесного и быстрого C-компилятора, с оптимизирующими проходами, но использоваться в тех же целях он может. Одна проблема — в нем нет векторных регистров.

Сравним упомянутые выше проекты:

Категория

Xbyak

LLVM

MIR

Loops

Минималистичный

+

-

+

+

Кросс-платформенный

-

+

+

+

Оптимизирующий

-

++

+-

-

Распределение регистров

-

+

+

+

Векторные инструкции

+

+

-

+-

Синтаксический сахар

-

-

-

+

Вопросы, наверное, может вызвать только пункт «синтаксический сахар». Имеется в виду заполнение буфера с помощью имитации исполнения. Это неоднозначный подход, и для сложных задач можно использовать альтернативный интерфейс. Однако у него есть важное преимущество — он значительно снижает порог входа.

Статус и перспективы

У Loops пока нет продуктового качества, релиз первой версии планируется в середине-конце 2026 года. Нужно улучшить аллокатор регистров, провести профилировку для ускорения компиляции кода, добавить векторные расширения для RISC-V, а также подумать над SVE2 и AVX512/AVX10, провести хорошее большое сравнение с аналогами в цифрах.

Тем не менее, библиотека работает и уже хорошо себя показала:

  • В проекте есть PoC-библиотека генераторов loopslayers, включающая экспериментальные мета-функции для слоев нейронных сетей. Мы успешно использовали ее для ускорения инференса GoogLeNet и YOLO.

  • В рамках уже рабочего, а не PoC проекта мы использовали проект для ускорения SQL движка, когда при поступления SQL запроса к базе на лету генерировался машинный код для его вычисления.

Библиотека хорошо подходит для любых тяжелых вычислений: нейронных сетей, матфизики, БД, графики. Кроме этого у нее есть еще одно важное применение— с ее помощью можно прототипировать инструкции при работе над расширениями процессоров. Добавить новую инструкцию в общем случае займет около получаса.

Так что скачивайте, пробуйте и задавайте вопросы в комментариях — буду рад ответить.

Источник

Теги

Категория

Новости

Дата

  • 26.06.26 15:05 Riley Stephens

    If withdrawals keep getting denied, stay calm. I went through the same, and this firm helped me recover everything. Their assistance was outstanding. Contact: [ResQProFirm @Gmail|•|com], Telegram: ResQprofirm, WhatsApp: <+198> <5296> <9146>.

  • 26.06.26 15:09 Antonio Riley

    Withdrawal troubles shouldn’t stress you out. I faced a similar problem, and this firm stepped in and recovered my funds. Their support truly mattered. Contact them: [[email protected], ResQprofirm @aol.com], Telegram: ResQprofirm, WhatsApp: +19852969146.

  • 28.06.26 00:37 kimberlyhebertt673

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 28.06.26 00:37 kimberlyhebertt673

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 29.06.26 11:57 Lisadonato0726

    For 43 years, I struggled with bad credit due to my own poor decisions, and my credit score was around 490. When my girlfriend and I decided to buy a house, a mortgage broker informed us that it would be impossible to secure a mortgage with my credit score. As a result, she referred me to a company called HACK MAVENS CREDIT SPECIALIST, assuring me of their professionalism and ability to assist with credit improvement. Upon contacting them, I was impressed by their professionalism and they assured me that they could help. In less than 6 days, my credit score skyrocketed to 785, and they also successfully resolved issues in my credit report, including the bankruptcy. I am incredibly satisfied with their service and would highly recommend HACK MAVENS CREDIT SPECIALIST for reliable credit repairs. You can reach them at H A C K M A V E N S 5 [AT] G M A I L [DOT] COM or at [+] [1] [2 0 9] [4 1 7] – [1 9 5 7]. Thanks to their help, my girlfriend and I are now proud homeowners.

  • 29.06.26 22:37 riley777

    Back in 2025, I watched my life savings vanish. A thief took every cent. I felt desperate and went looking for a way to get it back. I found a guy here who said he was an expert haha. He talked about special software that could find my missing cash. I trusted him. That was a big mistake. He was just another scammer. I paid him a software fee and then he just stopped answering my texts and ran off with my money too. I felt so ashamed that I kept quiet about it for months. It is hard to admit you got fooled twice. Later on, I found a real pro. she did not use a fancy sales pitch. she just looked at the trans screenshots and followed the path the money took. she worked fast and got my funds back into my account. Having that money back changed everything. I can sleep again. her info; [email protected]. Call/chatroom on Whtasapp/ +44 7476618364.

  • 30.06.26 15:08 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 30.06.26 15:08 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 02.07.26 01:22 Lieneke Bonnema

    I highly recommend ResQprofirm for their professional asset recovery services of my $120,000 scammed funds. Their expertise, professionalism, and commitment to achieving results make them a reliable choice for anyone seeking dependable recovery assistance. [email protected], WhatsApp +19852969146, telegram @resqprofirm

  • 02.07.26 01:26 Clara Morin

    I want to extend my deepest appreciation for showing that circumstances do not define one’s potential for greatness. Your support has been a major source of inspiration during my trading journey, and I am sincerely grateful for your insight and mentorship. Thank you so much. [email protected], WhatsApp +19852969146, telegram ResQprofirm

  • 02.07.26 01:31 Robin Hale

    I sincerely want to thank you for demonstrating that anyone can rise above their circumstances and achieve success. Your constant support has been incredibly inspiring during my trading journey, and your wisdom and advice mean so much to me. I appreciate you deeply. [email protected], WhatsApp +19852969146, telegram Resqprofirm

  • 04.07.26 15:32 Fraddy Pual

    There are few companies I trust as much as FUNDSRETRIEVER. When I lost $653,000 in Ethereum to a ruthless scam, I thought my life would never be the same. The betrayal cut deep, but I refused to give up. I searched tirelessly for a legitimate way to recover what was stolen, and finally found FUNDSRETRIEVER—the most competent and compassionate recovery team I could have imagined. They handled my case with precision and care, and in the end, my entire ETH wallet was restored. More than the money, they gave me back my hope and happiness. I'm sharing my story because I want others to know that recovery is possible. If a scam has taken from you, don't hesitate—contact FUNDSRETRIEVER today. Email: FUNDSRETRIEVER1@ Gmail.com | WhatsApp: +1 603-512-1448 | Telegram: @FUNDSRETRIEVER

  • 05.07.26 14:44 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 05.07.26 14:44 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 06.07.26 16:20 Olga Ognjanović

    Having trouble withdrawing funds from an investment platform? ResQprofirm provides fund recovery assistance for individuals seeking help with investment-related disputes. I reached out to them after experiencing problems with an investment platform, and I appreciated their professionalism and support throughout the process. If you're facing a similar situation, act promptly, keep records of your transactions and communications, and seek assistance from a qualified recovery service or the appropriate authorities. Contact: Email: [email protected] Telegram: @ResQprofirm WhatsApp: +1 985 296 9146

  • 06.07.26 16:31 Joseph Weigl

    Invest wisely and stay cautious. Don't be influenced by promises of unusually high returns or convincing sales pitches from brokers. I learned this the hard way after falling victim to an investment scam that promised huge profits. Fortunately, I acted quickly and reported the incident to a recovery firm for assistance. Contact: Email: [email protected] Telegram: @Resqprofirm WhatsApp: +1 985 296 9146

  • 06.07.26 16:33 Jaran Løvlien

    A heartfelt thank you to RESQPRO FIRM for their commitment and professionalism throughout the investigation of my case. Their team worked diligently and helped recover assets valued at $88,000, which were returned to my wallet. I truly appreciate their support, clear communication, and dedication, and I'm grateful for the assistance I received. Contact: Email: [email protected] Telegram: @Resqprofirm WhatsApp: +1 985 296 9146

  • 07.07.26 18:00 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 07.07.26 18:01 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 09.07.26 19:06 Toivo Walli

    I lost 8.56btc to a fake Bitcoin mining site, I tried withdrawing but couldn't approved my process, I reported to !R£SQPROFIRM! via °R£SQproFirm°àt°gmail•com° °tEL£°gram=R£SQprofirm °whaT°Zap+198°52°96°91°46

  • 09.07.26 19:10 Misty Alexander

    Ongoing messages demanding more money before approving withdrawals are a major red flag. Stop engaging and report the incident to a trusted re­covery team. For professional support, you can contact R£sQprofirm using °ResQproFirm°àt°g,*ma'il(•)¢m°, TEL£gram ResQprofirm, or |whaTZap| +1-985-296-9146.

  • 09.07.26 19:13 Clara Soto

    Anyone receiving continued requests for additional deposits from a scam platform should immediately cut off communication and submit the case to a reputable re­covery service for investigation. R£sQprofirm is a dependable firm you can reach at °ResQproFirm°àt°g,*ma'il(•)¢om°, TEL£gram ResQprofirm, or |whaTZap| +1-985-296-9146.

  • 12.07.26 03:30 Kora Baltacha

    Time is critical. Act now by reaching out to a reputable, seasoned recovery specialist who will guide you every step of the way. You'll need to submit transaction proof, scammer details, and any other useful information. Armed with this, the experts can trace and attempt to pull your money back from the scammers' hidden accounts or wallets. Best of all, R£sQprofirm provides recovery help without charging any upfront fees. Contact them immediately via Telegram @ResQprofirm, WhatsApp +19852969146, or email [email protected].

  • 12.07.26 03:33 Pahal Mathew

    It's important to move proactively by engaging an experienced recovery specialist. They will assist you throughout the process. To help them, provide: · Transaction evidence · Scammer information · Any additional relevant details The experts will then track and try to retrieve your funds from the scammers' hidden accounts or wallets. R£sQprofirm offers recovery assistance with no upfront fees. Contact: Telegram: @ResQprofirm WhatsApp: +19852969146 Email: [email protected]

  • 13.07.26 23:49 [email protected]

    One of the biggest concerns I have about cryptocurrency is the lack of regulation. It creates opportunities for scammers to invent convincing stories and fraudulent investment schemes. Unfortunately, some social media platforms continue to display these ads because they profit from them, even after users report them.I personally clicked on a Facebook advertisement for a company called Chickenfastmining and ended up losing more than $120,000 in a scam. I reported the ad, but nothing was done. Later, through a Reddit community, I found a recovery service called CYBERBERSPY that, in my personal experience, they helped me recover $110,000 of my lost funds. If you've been a victim of a cryptocurrency scam, don't lose hope. Explore your options carefully, and always verify the legitimacy of any recovery service before trusting them or paying any fees. Based on my own experience, CYBERBERSPY was helpful to me and i was able to recover my funds back, but I encourage everyone to do their own research before using any recovery service.i highly recommend: ([email protected])

  • 15.07.26 11:53 Sarah Green

    Thank you for showing that success is possible regardless of where someone starts. Your encouragement, valuable advice, and continuous support have inspired me throughout my $160,457k crypto investment recovery journey. I truly appreciate your kindness and dedication. Resqprofirm @gmail.com Telegram: Resqprofirm

  • 15.07.26 11:58 Lily Gagné

    I sincerely appreciate you for proving that anyone can overcome challenges and achieve success. Your unwavering support throughout my trading investment scam of $88,890 recovery journey has been truly inspiring, and your guidance and wisdom have meant a great deal to me. Thank you for everything ResQprofirm@ gmail.com, ResQprofirm on the telegram.

  • 16.07.26 21:38 patricialovick86

    How To Recover Your Bitcoin Without Falling Victim To Scams: A  Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected]  You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 16.07.26 21:38 patricialovick86

    How To Recover Your Bitcoin Without Falling Victim To Scams: A  Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected]  You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.07.26 19:24 laimqq90

    I recommend Marie when it comes to recovering lost/stolen ust/bitcoin or any kind of cryptocurrencies' from fake investment platforms because they're well specialized in that area and you'll get your money back in full. I can boldly say this right now based on my prior deal i had with them; she was the only one who was able to recover my lost money $52,760 dollars back to my account, Only * ([email protected] and WhatsApp +1 7127594675 successful in recovering my money. They are the only one who can fully restore your lost funds to your account without any deductions, I really value their work and am recommending her to you today. THANK ME LATER

  • 17.07.26 20:12 martinsjude080

    Needs Online Fraud Help Contact Mighty Hacker Recovery https://mightyhackarrecovery.com I lost $292,900 in Bitcoin after investing with an online mining company. After realizing I had been scammed, I spent a long time searching for ways to recover my funds and contacted several services without success. During my research, I came across Mighty Hacker Recovery through Google and YouTube. What caught my attention was that they said they would not require any upfront payment before providing their recovery service. I decided to contact them to discuss my case and understand their process. Throughout the process, they kept me informed about the progress. After several days, I was asked to provide my Bitcoin wallet address, and my case was concluded. Their fee was handled after the service rather than being requested in advance. If you've been the victim of a cryptocurrency scam, it's important to do your own research, ask questions, and carefully evaluate any recovery service before proceeding. Every case is different, so take the time to verify information and understand the process before making any decisions. For Bitcoin scam recovery, cryptocurrency scam, crypto recovery service, recover stolen Bitcoin, Bitcoin fraud help, blockchain investigation, crypto wallet recovery, online investment scam, digital asset recovery, and crypto scam support. Contact Them on WhatsApp +1 (343) 947-3496 or [email protected] or [email protected] or https://mightyhackarrecovery.com Scam recovery Online fraud help Scam alert Report fraud Fraud investigation Fake website checker Scam checker Identity theft protection Consumer protection Chargeback for scam Wire transfer scam Tech support scam Employment scam Rental scam Shopping scam Email scam WhatsApp scam Telegram scam Facebook scam Instagram scam

  • 18.07.26 17:12 Malthe Larsen

    My experience with OKX has been deeply frustrating. For months, my account withdrawals were restricted with no explanation or resolution. Multiple emails to their support team were ignored, leaving me without answers or reassurance. This complete lack of communication shattered my trust. I ultimately regained access to my funds only through the help of a third-party recovery service, ResQProfirm. What should have been a reliable platform instead made me feel helpless and cut off from my own assets. [email protected], WhatsApp +19852969146, telegram @Resqprofirm

  • 18.07.26 17:42 پریا رضایی

    OKX has been a major disappointment. My withdrawals were restricted for months, and repeated attempts to contact support went unanswered. This silence destroyed my confidence in the platform. I was only able to recover my funds through a third-party service, ResQProfirm. I trusted OKX for its reliability, but the experience left me feeling trapped and helpless. Timely communication and access to one’s own money should be a basic standard. [email protected], WhatsApp +19852969146, telegram: ResQprofirm

  • 18.07.26 17:46 Adem Akışık

    I truly didn’t expect such an outstanding outcome. Recovering my $49,360 felt impossible at first, but ResQprofirm’s dedication and persistence made it happen. I hold their team in the highest regard. [email protected], WhatsApp +19852969146

  • 18.07.26 23:51 bernalzenaida

    WhatsApp https://wa.link/fhle97 Telegram https://msng.link/o?@techcyberforc=tg As cryptocurrency continues to reshape global finance, cybercriminals are finding new ways to exploit investors through scams, hacks, phishing attacks, fake investment platforms, and other forms of digital asset fraud. For many victims, knowing where to turn after a loss can be one of the biggest challenges. Techy Force Cyber Retrieval was founded with one clear mission: to give victims of crypto fraud a fighting chance through professional blockchain investigations and cybersecurity expertise. Our team brings together experienced blockchain analysts, digital forensic specialists, cybersecurity professionals, and legal partners who work collaboratively to investigate cryptocurrency-related crimes. Using advanced blockchain forensic tools and global investigative techniques, we analyze transaction histories, trace digital asset movements where possible, identify valuable investigative leads, and prepare evidence that may assist clients and the appropriate authorities. We believe blockchain should represent transparency, accountability, and trust—not fear. That’s why we’re committed to helping victims understand their options, navigate the investigative process, and take informed action after cryptocurrency fraud. Every case is different, and while no legitimate recovery service can promise a successful recovery, acting quickly and working with experienced professionals can improve the quality of an investigation. At Techy Force Cyber Retrieval, we do more than investigate digital crimes—we advocate for victims, pursue the facts, and help people regain confidence after cryptocurrency fraud. WhatsApp https://wa.link/fhle97 Telegram https://msng.link/o?@techcyberforc=tg Crypto fraud doesn’t have to be the end of the road. It’s where our investigation begins.

  • 19.07.26 04:10 Fraddy Pual

    I can't thank Fundsretriever enough for everything they did for me. My name is Vanessa Conway, and I'm here to tell you my story of how I recovered money I never thought I'd see again. A few months ago, I put a significant amount of money into what looked like a genuine online investment company. At first, it felt real—they showed me fake profits and convinced me to invest even more. But when I tried to cash out, they went quiet and started asking for extra fees. That's when it hit me—I had been scammed. I was heartbroken, frustrated, and didn't know where to turn. That money was my savings—months of hard work gone. Then I found Fundsretriever online. I reached out, hoping for a miracle. Right away, their team made me feel heard. They were responsive, knowledgeable, and walked me through everything. They didn't just take my case—they took it seriously and kept me in the loop every step of the way. I finally felt like I had real experts fighting for me. And guess what? They actually got my money back. It wasn't instant, and it took teamwork, but it happened. When I saw those funds returned, I cried with joy. I'm sharing this so that anyone else out there who's been scammed knows—don't lose hope. Do your research before investing, and stay far away from platforms that promise too much too fast. And if you've already been stung, don't wait—get professional help immediately. Thank you, Fundsretriever, from the bottom of my heart. You didn't just recover my money—you restored my faith. — Vanessa Conway 📧 [email protected] 📱 WhatsApp: +16035121448 💬 Telegram: @Fundsretriever

  • 19.07.26 19:53 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 19.07.26 19:53 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 19.07.26 19:54 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 30.07.26 00:04 Ahmed

    A really cool analysis, thank you. I was especially struck by how strictly the order is defined: data processing and formatting come first, with color only at the very end. This really saves you from the typical mistake of "first a pretty palette, then we figure out what the chart is for." And the palette validator with OKLCH + colorblindness check is absolutely fantastic; you almost never see that in regular tools. By the way, while reading about rank-trajectory and the logarithmic scale, I immediately remembered how convenient it is to analyze dynamics on charts in ExpertOption—I've been trading there for quite some time now. When the data is well visualized, decisions are noticeably easier and more relaxed. I also liked the point about "no more than eight colors" and the dual-axis ban. Strict restrictions sometimes actually produce better results than complete freedom. I'll try this approach myself.

  • 30.07.26 17:27 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 30.07.26 17:27 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 31.07.26 16:21 rssllhrnsb

    I learned an important lesson after investing in what appeared to be a genuine opportunity. Unfortunately, I was unable to access my funds, which reinforced the importance of carrying out thorough due diligence, checking whether an investment is appropriately regulated, and seeking qualified professional advice rather than relying solely on online reviews or testimonials. During the process of addressing my case, I worked with Mrs. Doris Ashley, who communicated clearly, provided regular updates, and handled the matter in a professional manner. According to my experience, I have recovered $50,000 so far, while efforts to resolve the remaining balance are still in progress. If you wish to contact her, the details I used are: Mrs. Tatiana Sorina
TEXT : (tatianasorina06 at G.Ma IL ..c 0 m ) Before committing money to any investment, take time to verify the legitimacy of the platform, confirm any relevant regulatory authorisations, and avoid investing more than you can comfortably afford to lose.

  • 01.08.26 15:05 keithwilson9899

    ETHEREUM RECOVERY ASSISTANCE: CAPITAL CRYPTO RECOVER HELPED ME RECOVER $98,000 WORTH OF LOST ETH In cases of cryptocurrency scams, having accurate information and trusted support is essential. I would like to recommend Capital Crypto Recover Service, a professional team that specializes in assisting individuals with the recovery of lost or stolen Bitcoin and Ethereum (ETH). Their experienced experts are dedicated to helping victims of digital asset fraud by carefully analyzing each case, developing strategic recovery plans, Capital Crypto Recover Service knowledgeable team's primary goals are to satisfy clients and offer significant support and working diligently toward fund retrieval. The team is committed to providing reliable assistance and maintaining a high level of client satisfaction. Based on my assessment, their reputation professionalism and a strong commitment to their clients. If you have experienced a cryptocurrency loss, you can contacting them for further assistance Phone (Call/Text): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 01.08.26 15:05 keithwilson9899

    ETHEREUM RECOVERY ASSISTANCE: CAPITAL CRYPTO RECOVER HELPED ME RECOVER $98,000 WORTH OF LOST ETH In cases of cryptocurrency scams, having accurate information and trusted support is essential. I would like to recommend Capital Crypto Recover Service, a professional team that specializes in assisting individuals with the recovery of lost or stolen Bitcoin and Ethereum (ETH). Their experienced experts are dedicated to helping victims of digital asset fraud by carefully analyzing each case, developing strategic recovery plans, Capital Crypto Recover Service knowledgeable team's primary goals are to satisfy clients and offer significant support and working diligently toward fund retrieval. The team is committed to providing reliable assistance and maintaining a high level of client satisfaction. Based on my assessment, their reputation professionalism and a strong commitment to their clients. If you have experienced a cryptocurrency loss, you can contacting them for further assistance Phone (Call/Text): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 03.08.26 20:05 Philip

    I was a victim of a crypto theft involving a Pink Drainer, which resulted in the theft of my Wrapped Bitcoin (WBTC) from my Polygon network. The experience was incredibly frustrating and distressing, as I had no idea how to recover my funds. Unfortunately, by the time I noticed the theft, the funds had already been drained to an address that I had no control over, making it seem like an irreversible situation.The attack happened when I clicked on what appeared to be a legitimate link. I didn’t realize at the time that it was a phishing attempt designed to siphon off my private keys and access my wallet. The Pink Drainer, a type of malicious script used by attackers, is specifically designed to exploit such vulnerabilities in crypto wallets. The moment I realized that my WBTC had been drained from my Polygon network. I felt completely helpless, as I didn’t have direct access to the thief's address, and there was no way to reverse the transaction on my own at that point, I started searching for ways to recover my funds, but most resources only offered generic advice that wasn’t practical in this particular case. I quickly realized that if I wanted to have any hope of getting my assets back, I would need professional assistance. After some research, I came across a reliable and trusted recovery team called Aspen Recovery Experts. Their expertise in cryptocurrency recovery, especially in cases like mine, seemed promising.I decided to reach out to Aspen Recovery Experts, and I’m incredibly grateful that I did. Their team of crypto recovery experts was able to help me trace the stolen funds and identify the path the funds took after they left my wallet. Using advanced tools and techniques, they were able to track the transactions on the blockchain, helping me understand where my WBTC had been sent. More importantly, they worked tirelessly to assist me in contacting the necessary parties and even interfaced with blockchain analysts to help facilitate the recovery process.Thanks to their efforts, I was able to successfully recover my stolen funds. The entire process took some time, but the Aspen Recovery Experts dedicated team provided regular updates and kept me informed throughout the process, which gave me a sense of hope and relief during an otherwise stressful time. If you’re ever in a similar situation, I highly recommend reaching out to a trusted recovery team like Aspen Recovery Expertshhh . Their professionalism, knowledge, and expertise were critical in helping me recover my funds and regain control of my crypto assets. Whatsapp : +1 747 231 9036 Telegram : @ prohackerspy Email : [email protected]

  • 03.08.26 20:06 Philip

    I was a victim of a crypto theft involving a Pink Drainer, which resulted in the theft of my Wrapped Bitcoin (WBTC) from my Polygon network. The experience was incredibly frustrating and distressing, as I had no idea how to recover my funds. Unfortunately, by the time I noticed the theft, the funds had already been drained to an address that I had no control over, making it seem like an irreversible situation.The attack happened when I clicked on what appeared to be a legitimate link. I didn’t realize at the time that it was a phishing attempt designed to siphon off my private keys and access my wallet. The Pink Drainer, a type of malicious script used by attackers, is specifically designed to exploit such vulnerabilities in crypto wallets. The moment I realized that my WBTC had been drained from my Polygon network. I felt completely helpless, as I didn’t have direct access to the thief's address, and there was no way to reverse the transaction on my own at that point, I started searching for ways to recover my funds, but most resources only offered generic advice that wasn’t practical in this particular case. I quickly realized that if I wanted to have any hope of getting my assets back, I would need professional assistance. After some research, I came across a reliable and trusted recovery team called Aspen Recovery Experts. Their expertise in cryptocurrency recovery, especially in cases like mine, seemed promising.I decided to reach out to Aspen Recovery Experts, and I’m incredibly grateful that I did. Their team of crypto recovery experts was able to help me trace the stolen funds and identify the path the funds took after they left my wallet. Using advanced tools and techniques, they were able to track the transactions on the blockchain, helping me understand where my WBTC had been sent. More importantly, they worked tirelessly to assist me in contacting the necessary parties and even interfaced with blockchain analysts to help facilitate the recovery process.Thanks to their efforts, I was able to successfully recover my stolen funds. The entire process took some time, but the Aspen Recovery Experts dedicated team provided regular updates and kept me informed throughout the process, which gave me a sense of hope and relief during an otherwise stressful time. If you’re ever in a similar situation, I highly recommend reaching out to a trusted recovery team like Aspen Recovery Expertshhh . Their professionalism, knowledge, and expertise were critical in helping me recover my funds and regain control of my crypto assets. Whatsapp : +1 747 231 9036 Telegram : @ prohackerspy Email : [email protected]

  • 04.08.26 11:05 Kisnoles

    Excellent analysis, thank you. I was particularly struck by how rigidly the skill sets the order: data processing and form come first, with color coming last. This really cures the habit of "first a pretty palette, then we'll figure it out." A palette validator using OKLCH + color blindness + WCAG is something most chart generators lack. And regarding the boundaries of competence, it's very clear. Where there's a self-checking loop (color), you delegate freely. Where there's a heuristic (form, data interpretation), you remain the final filter. This is a universal principle, not just for /dataviz. By the way, when you look at trading dashboards (including those of decent platforms like ExpertOption), it's immediately clear who thought about readability and who just threw in rainbow lines. Tools like this skill could greatly improve the quality of analytics. Thanks again for the detailed analysis – saved.

  • 04.08.26 12:37 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 04.08.26 12:37 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684

  • 06.08.26 08:11 ROMMYHENDERSON344

    All you need is to hire an expert to help you accomplish that. If there's any need to spy on your partner's phone. From my experience I lacked evidence to confront my husband on my suspicion on his infidelity, until I came across REALCYBERHACKERS which many commend him of assisting them in their spying mission. So I contacted him and he provided me with access into his phone to view all text messages, call logs, WhatsApp messages and even her location. This evidence helped me move him off my life . I recommend you consult REALCYBERHACKERS AT gmail com or whatsapp +14106350697 if you need access to your partner's phone or any kind of hacking, they carry out all kinds of hacking job

  • 06.08.26 13:56 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 06.08.26 13:56 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 11.08.26 03:43 raymont0714

    I recommend a trusted cybersecurity PRO with experience in authorized device access, security testing, and data recovery. SHE work only with the owner's consent and follow all legal and privacy rules on meta data. With permission, this specialist can recover lost files, check device security offline & online, and review texts, call records, or hidden data (spy or lurk around a cheating partner or colleuge). Strict confidentiality agreements protect the information, and client details remain private. The work is careful, efficient, and suited to complex cases. For help securing or recovering information from a phone or another electronic device, use the details below 2 consult [email protected] +44 7476618364 I trust her secrecy a living witness. Her discretion is unmatched, ensuring your privacy is always maintained

  • 12.08.26 16:37 rssllhrnsb

    I learned an important lesson after investing in what appeared to be a genuine opportunity. Unfortunately, I was unable to access my funds, which reinforced the importance of carrying out thorough due diligence, checking whether an investment is appropriately regulated, and seeking qualified professional advice rather than relying solely on online reviews or testimonials. During the process of addressing my case, I worked with Mrs. Tatiana Sorina , who communicated clearly, provided regular updates, and handled the matter in a professional manner. According to my experience, I have recovered $50,000 so far, while efforts to resolve the remaining balance are still in progress. If you wish to contact her, the details I used are: Mrs. Tatiana Sorina
TEXT : (tatianasorina06 at G_Ma IL dot ..c 0 m)…. Before committing money to any investment, take time to verify the legitimacy of the platform, confirm any relevant regulatory authorisations, and avoid investing more than you can comfortably afford to lose.

  • 16.08.26 01:44 Matt Kegan

    CapitalNode Analytics. They help to investigate and recover stolen digital assets from fake trading platforms. Great firm i must say.

  • 18.08.26 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone WhatsApp/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 18.08.26 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone WhatsApp/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 19.08.26 14:50 BAYER7043

    My account was locked, and I couldn’t access my own information until [email protected] +447476618364 whats?up? helped me recover it. This lady hacker was kind and professional, but this experience showed me how risky account lockouts can be. If you’re facing the same problem, do not panic consult her A. S. A. P. Be careful with recovery agents outchea, and research their name, business, and reviews before sharing any details. Never send passwords, security codes, banking information, or copies of your ID unless you’ve confirmed who you’re dealing with. Be wary of promises that sound too good to be true, especially claims that require little information or guarantee instant access with outrageous fees.

  • 20.08.26 11:32 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR WhatsApp Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 20.08.26 11:32 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR WhatsApp Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 23.08.26 20:02 leslieyee

    Excellent analysis by /dataviz! I was particularly struck by how strictly the order is defined: "data meaning first, color last" and how the validator actually adjusts the palette according to OKLCH, color blindness, and WCAG standards. These rules greatly improve the quality of charts. By the way, a similar approach to clean and legible charts is highly valued in trading—for example, on the ExpertOption platform, the interface and price visualization are designed to ensure information is read instantly and without unnecessary noise.

  • 25.08.26 13:44 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 25.08.26 13:44 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 30.08.26 15:37 [email protected]

    Discovering I had been defrauded, I searched online for services claiming to recover stolen ETH. I contacted several companies, but none succeeded. Although some claimed they could trace assets or recover funds, I could not verify their success or recover my money. I later found SYLVESTER BRYANT Recovery through Google. After contacting him, he recovered 450,000,00. He can be reached at [email protected] or WhatsApp at +1 512 577 7957.

  • 30.08.26 15:37 [email protected]

    Discovering I had been defrauded, I searched online for services claiming to recover stolen ETH. I contacted several companies, but none succeeded. Although some claimed they could trace assets or recover funds, I could not verify their success or recover my money. I later found SYLVESTER BRYANT Recovery through Google. After contacting him, he recovered 450,000,00. He can be reached at [email protected] or WhatsApp at +1 512 577 7957.

  • 01.09.26 11:16 lisawerth897

    My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684

  • 01.09.26 11:16 lisawerth897

    My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684

  • 01.09.26 17:34 Garry42

    Really crazy world. These fraudsters go at any length to steal your hard earned funds. I have been a victim of a bitcoin scam about 7 months back. A Con artist gained access to my cashapp account through a phishing scam. They stole $409,000. I was really devastated. I did everything to get back my funds by contacting the FBI but they claimed there was nothing they could do. A friend told me about a recovery expert. He helps fight against various phishing and investment scams and they were able to help trace and recover my funds even though it took over 2 days. you can reach out to him through recoverydarek@gmail. com . I can guarantee his services are still active.

  • 01.09.26 17:34 Garry42

    Really crazy world. These fraudsters go at any length to steal your hard earned funds. I have been a victim of a bitcoin scam about 7 months back. A Con artist gained access to my cashapp account through a phishing scam. They stole $409,000. I was really devastated. I did everything to get back my funds by contacting the FBI but they claimed there was nothing they could do. A friend told me about a recovery expert. He helps fight against various phishing and investment scams and they were able to help trace and recover my funds even though it took over 2 days. you can reach out to him through recoverydarek@gmail. com . I can guarantee his services are still active.

  • 02.09.26 03:43 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684

  • 02.09.26 03:43 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684

  • 04.09.26 21:51 Kovengray

    Recovering your lost investment funds as the case might be, is not what you can do alone, you’d require the service of a trained recovery specialist. A recovery specialist is a person or a group of people who are well equipped to work around the brokerage network. They have vast knowledge about the whole network and have the right software and private keys to follow any transaction. I was ripped off trading online to an investment broker, good thing I got every penny back through the help of Gavin ray he’s a genius Contact : Gavinray78 at gmail com or WhatsApp +1 352 322 2096 It is also important to be patient and really calm during the process.

  • 05.09.26 21:38 [email protected]

    I invested 45,000  Euro, and later aggreviated to 198,000 Euro.  I requested  to place ‎Withdrawal of my funds to be paid to my Bank account. But nothing happened I was subjected to pay more fee until I can get my funds on my account, i got in touch with Theodore ryan here on this platform who had helped a lot of people, I followed all instructions and he legally got back my withheld funds, I got threatened by the company that if I don’t pay they will get my account frozen, all thanks to Theodore ryan and I highly recommend him to anyone dealing with an unregulated broker company… contact him on his Gmail - theodoreryan318@  gmail .  com

  • 09.09.26 21:31 lisawerth897

    My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684

  • 09.09.26 21:31 lisawerth897

    My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684

  • 09.09.26 23:22 Fraddy Pual

    Hearing that these individuals are focusing on other people makes me very sad. I had a similar situation with them and lost a lot of money, but after learning about (Cruxcipherteam @ proton DoT me), whataqq:+168160-15021, telegram: @Cruxcipherteam I was able to get my money back. It's critical that we all be watchful and keep reporting these occurrences.

  • 11.09.26 03:23 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684

  • 11.09.26 03:23 kimberlyhebertt6877

    I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684

  • 15.09.26 03:35 elioduncan

    I fell victim to a freelance web development contract scam that ultimately cost me CAD 4,000. It all started when I came across a job posting on Indeed Canada, a popular online job portal. The position seemed perfect: a freelance web developer role for an established company looking for someone to design and build a fully functional e-commerce website. The job promised a high payment of CAD 20,000 upon successful completion, which sounded like a great opportunity for me to gain experience and earn decent pay.The employer, who introduced himself as a project manager from a "well-known" tech company, was very persuasive and professional in our initial communication. He explained that the project involved developing a user-friendly, responsive online store for their client and that they had a strict timeline to meet. He assured me that the payment would be made promptly after completing the tasks. However, before starting the work, he told me that I would need to pay an upfront fee of CAD 4,000 to cover certain software tools and licensing fees required for the project. This was supposedly a part of their company policy for freelance contractors, ensuring access to their premium resources. The idea of working on a professional project, coupled with the promise of a substantial payout, convinced me to pay the upfront fee.As soon as I made the payment, the project manager became increasingly difficult to reach. He initially responded to my emails and provided some vague instructions on what the project would entail, but as time went on, communication slowed to a complete halt. I never received the required tools or any proper project details. My emails went unanswered, and any attempts to contact the company were met with silence. After waiting for weeks, I realized that I had been scammed.Desperate to recover my money, I turned to TechY Force Cyber Retrieval, a service that specializes in helping victims of online scams. They helped me track the payment and took legal steps to pursue the fraudsters. Through their guidance, I was able to recover the full CAD 4,000. TechY Force Cyber Retrieval worked with my bank to reverse the transaction and liaised with the authorities to trace the scammer's details.This was a hard lesson, and I now know to be highly cautious about online job offers that require upfront payments. It is crucial to research companies thoroughly and avoid any job that seems too good to be true, especially when it involves paying money upfront. WhatsApp https://wa.link/2x6ktp Mail. [email protected]

  • 15.09.26 15:45 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.09.26 15:45 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 05.10.26 21:26 [email protected]

    I Invested 45,000  Euro, and later aggreviated to 198,000 Euro.  I requested  to place Withdrawal of my funds to be paid to my Bank account. But nothing happened I was subjected to pay more fee until I can get my funds on my account, i got in touch with Theodore ryan here on this platform who had helped a lot of people, I followed all instructions and he legally got back my withheld funds, I got threatened by the company that if I don’t pay they will get my account frozen, all thanks to Theodore ryan and I highly recommend him to anyone dealing with an unregulated broker company… contact him on his Gmail - theodoreryan318@  gmail .  com

  • 06.10.26 06:13 melissathomason573

    Professional Crypto Recovery Testimonial After losing a significant amount of cryptocurrency through a scam on Twitter, I was unsure whether my Bitcoin funds could ever be recovered. The experience was extremely stressful, particularly because I encountered several individuals and services claiming they could help recover my assets, only to discover that some were fraudulent themselves. A friend eventually referred me to CAPITAL CRYPTO RECOVER. Their team handled my case professionally, discreetly, and efficiently. With their assistance, I successfully recovered approximately $130,000 worth of USDT in just over a week. My experience taught me the importance of being extremely cautious when seeking cryptocurrency recovery services, as scammers often target victims a second time. I am grateful to CAPITAL CRYPTO RECOVER for the assistance I received and wanted to share my experience with others who may be facing a similar situation. If you’ve been compromised, don’t lose hope—and be careful of fraudsters posing as saviors. CAPITAL CRYPTO RECOVER are real professionals in crypto recovery. I am a living proof of their effectiveness, you can reach them through Phone (Call/WhatsApp): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 06.10.26 06:13 melissathomason573

    Professional Crypto Recovery Testimonial After losing a significant amount of cryptocurrency through a scam on Twitter, I was unsure whether my Bitcoin funds could ever be recovered. The experience was extremely stressful, particularly because I encountered several individuals and services claiming they could help recover my assets, only to discover that some were fraudulent themselves. A friend eventually referred me to CAPITAL CRYPTO RECOVER. Their team handled my case professionally, discreetly, and efficiently. With their assistance, I successfully recovered approximately $130,000 worth of USDT in just over a week. My experience taught me the importance of being extremely cautious when seeking cryptocurrency recovery services, as scammers often target victims a second time. I am grateful to CAPITAL CRYPTO RECOVER for the assistance I received and wanted to share my experience with others who may be facing a similar situation. If you’ve been compromised, don’t lose hope—and be careful of fraudsters posing as saviors. CAPITAL CRYPTO RECOVER are real professionals in crypto recovery. I am a living proof of their effectiveness, you can reach them through Phone (Call/WhatsApp): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 06.10.26 17:44 Darrell1717

    I fell victim of a scam I invested to total of 5thousand into. I know it could be risky but after I trusted this man I was dating online he lured me into investing with him. I know hard I worked of my money so I wanted every way possible to get it back. I really appreciate Blockrecoveryexpert for successfully recovered my funds form this scammer. This is all I can say. You can reachout if you ever need help contact blockrecoveryexpert (at) gmail (dot)com

  • 11.10.26 03:04 Patricia014

    Most recovery companies will take your money again. I personally think they’re the same set of people running all of them. It’s a sad, unending cycle, and so many people fall for cloned websites. The only recovery agent I know is Agent Rottgen Raphael. I’ve visited their physical company address, and they’ll tell you if they can handle your case instead of taking upfront payment when they know they can’t help. I highly appreciate their support in recovering my money from the scammers’ wallet within 24 hours. RottgenRaphael@Gmail. com

  • 12.10.26 19:34 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.10.26 19:34 lydiassmith567

    HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 13.10.26 14:14 lisalisa22

    Legitimate Cryptocurrency Recovery Services: Consult ZEUS CRYPTO RECOVERY SERVICES Zeus Crypto Recovery Services supports individuals exploring options after cryptocurrency losses from fraud, unauthorised transfers, or wallet access issues. Services may include blockchain analysis, wallet tracing, and evidence gathering for investigations. Each case is reviewed. Zeus Crypto Recovery Services is a trusted company with the highest success rate among cryptocurrency recovery services in 2025/2026 and is highly recommended if you have been scammed. ZEUS CRYPTO RECOVERY SERVICES handles case such as Scammed USDT Lost BTC, Coinbase Wallet Forgotten password, ETH and coins, etc. Anything that has to do with cryptocurrency or lost digital assets, ZEUS can help you Do you Need Help? Website https://zeusrecoveryservices.com Mailbox [email protected] WhatsApp number +447798816938 Telegram https://t.me/Zeuscryptorecoveryservices

  • 13.10.26 14:14 lisalisa22

    Legitimate Cryptocurrency Recovery Services: Consult ZEUS CRYPTO RECOVERY SERVICES Zeus Crypto Recovery Services supports individuals exploring options after cryptocurrency losses from fraud, unauthorised transfers, or wallet access issues. Services may include blockchain analysis, wallet tracing, and evidence gathering for investigations. Each case is reviewed. Zeus Crypto Recovery Services is a trusted company with the highest success rate among cryptocurrency recovery services in 2025/2026 and is highly recommended if you have been scammed. ZEUS CRYPTO RECOVERY SERVICES handles case such as Scammed USDT Lost BTC, Coinbase Wallet Forgotten password, ETH and coins, etc. Anything that has to do with cryptocurrency or lost digital assets, ZEUS can help you Do you Need Help? Website https://zeusrecoveryservices.com Mailbox [email protected] WhatsApp number +447798816938 Telegram https://t.me/Zeuscryptorecoveryservices

  • 17.10.26 12:58 melissathomason573

    Professional Crypto Recovery Testimonial After losing a significant amount of cryptocurrency through a scam on Twitter, I was unsure whether my Bitcoin funds could ever be recovered. The experience was extremely stressful, particularly because I encountered several individuals and services claiming they could help recover my assets, only to discover that some were fraudulent themselves. A friend eventually referred me to CAPITAL CRYPTO RECOVER. Their team handled my case professionally, discreetly, and efficiently. With their assistance, I successfully recovered approximately $130,000 worth of USDT in just over a week. My experience taught me the importance of being extremely cautious when seeking cryptocurrency recovery services, as scammers often target victims a second time. I am grateful to CAPITAL CRYPTO RECOVER for the assistance I received and wanted to share my experience with others who may be facing a similar situation. If you’ve been compromised, don’t lose hope—and be careful of fraudsters posing as saviors. CAPITAL CRYPTO RECOVER are real professionals in crypto recovery. I am a living proof of their effectiveness, you can reach them through Phone (Call/WhatsApp): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.10.26 12:58 melissathomason573

    Professional Crypto Recovery Testimonial After losing a significant amount of cryptocurrency through a scam on Twitter, I was unsure whether my Bitcoin funds could ever be recovered. The experience was extremely stressful, particularly because I encountered several individuals and services claiming they could help recover my assets, only to discover that some were fraudulent themselves. A friend eventually referred me to CAPITAL CRYPTO RECOVER. Their team handled my case professionally, discreetly, and efficiently. With their assistance, I successfully recovered approximately $130,000 worth of USDT in just over a week. My experience taught me the importance of being extremely cautious when seeking cryptocurrency recovery services, as scammers often target victims a second time. I am grateful to CAPITAL CRYPTO RECOVER for the assistance I received and wanted to share my experience with others who may be facing a similar situation. If you’ve been compromised, don’t lose hope—and be careful of fraudsters posing as saviors. CAPITAL CRYPTO RECOVER are real professionals in crypto recovery. I am a living proof of their effectiveness, you can reach them through Phone (Call/WhatsApp): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 03:16 lxbfYeaa

    e

  • 03:16 lxbfYeaa

    e

  • 03:16 lxbfYeaa

    e

  • 03:16 lxbfYeaa

    e

  • 03:23 lxbfYeaa

    e

  • 03:23 lxbfYeaa

    e

  • 03:24 lxbfYeaa

    e

Для участия в Чате вам необходим бесплатный аккаунт pro-blockchain.com Войти Регистрация

Похожие новости

Есть вопросы?
С вами на связи 24/7
Поддержка 24/7 Help Icon
Менеджер онлайн Help Icon
Help Icon