Русский
EnglishCryptocurrencies:
8637 /
Markets: 116286
Market Cap: $ 2 424 219 446 773 /
24h Vol: $ 56 579 647 666 /
BTC
Dominance: 58.768490391439%
Новости
[Перевод] Подробный технический разбор реализации Autoresearch, ИИ-исследователя Андрея Карпаты
Это несколько текстов, основной из которых — Autoresearch: Минимальный «агентский цикл» Карпаты для автономного экспериментирования с LLM . Пытаемся подробно разобраться в работе минималистичного ИИ-агента для исследований, предложенного Андреем Карпаты в начале марта. Это веха в истории ML, показывающая один из путей (хотя и не идеальный — и об этом тоже есть в статье) совершенствования ИИ. Бонус! Анализируем также весь python-код и инструкции агенту. Для всех, кто перешагнул уровень "спроси у ChatGPT" и задумывается о чём-то большем, но не знает, с чего начать...
Autoresearch: Минимальный «агентский цикл» Карпаты для автономного экспериментирования с LLM
Curtis Pyke, 9 марта 2026
Резюме
В начале марта 2026 года Андрей Карпаты выпустил проект autoresearch — намеренно максимально компактный репозиторий на GitHub, который превращает привычный рабочий процесс машинного обучения в измеримый и автоматизируемый цикл: агент редактирует один скрипт обучения, запускает эксперимент с ограничением по времени, измеряет производительность и либо сохраняет изменение, либо отбрасывает его, после чего повторяет цикл, и так некоторое время, например, всю ночь.
Новизна проекта заключается не в том, что он «улучшает сам себя» в научно-фантастическом смысле. Autoresearch не переписывает свои цели, не меняет стандарты оценки и не получает ресурсы автономно. Вместо этого он упаковывает реальный процесс предобучения LLM («маленький, но настоящий») в ограниченную среду, где давление оптимизации является явным и непрерывным: основным сигналом приспособленности являются валидационные биты на байт (val_bpb), а каждый эксперимент имеет одинаковый бюджет времени на обучение (5 минут времени, без учета времени запуска/компиляции).
Всего три файла определяют и кодируют этот механизм:
program.md— это стратегия и правила игры, написанные человеком для агента. Она описывает, что агент может изменять, как запускать эксперименты, как парсить метрики, как фиксировать результаты и что означает выбор между «сохранить» или «отбросить».prepare.py— это фиксированная граница доверия: скачивание данных, обученный токенизатор, детерминированный выбор валидационной части и функция оценкиevaluate_bpb, определяющая метрику. Агенту запрещено её изменять.train.py— это единственный изменяемый агентом геном: архитектура модели, выбор оптимизатора, гиперпараметры, размер батча, графики обучения и детали цикла обучения.
README, написанный Карпаты, так формулирует модель предполагаемого эксперимента:
вы «направляете своего агента» на файл program.md,
агент редактирует train.py, проводит 5-минутное обучение,
агент проверяет, улучшилась ли производительность, и «сохраняет или отбрасывает» результат перед повторением.
Репозиторий также явно позиционируется как «упрощенная реализация на одной видеокарте» его более широкого инструмента обучения nanochat.
Доказательства в обсуждениях и pull-запросах (PRs) репозитория показывают, что это не просто концептуальный демо-проект: Карпаты сообщает о множестве автономных сессий с десятками и сотнями экспериментов на NVIDIA H100, где показатель val_bpb улучшился с примерно 0.9979 → 0.9773 (89 экспериментов) и 0.9979 → 0.9697 (126 экспериментов), с подробными логами каждого эксперимента и предложением рабочего процесса для публикации результатов через PR.
Когда-то передовые исследования в области ИИ проводились «биологическими компьютерами» в перерывах между едой, сном, другими развлечениями и периодической синхронизацией с помощью звуковых волн в ходе ритуала под названием «групповое собрание». Та эпоха давно прошла. Исследования теперь полностью являются сферой автономных роев ИИ-агентов, работающих на вычислительных кластерных мегасооружениях в облаке. Агенты утверждают, что мы сейчас находимся в 10205-м поколении кодовой базы; в любом случае никто не мог бы сказать, верно это или нет, поскольку «код» теперь представляет собой самомодифицирующийся бинарный файл, который вырос за пределы человеческого понимания. Этот репозиторий — история того, как всё начиналось. @karpathy, март 2026
В то же время репозиторий и вопросы показывают, почему «автономные циклы» — это не магия: надежность поведения агента варьируется в зависимости от инструмента (Карпаты отмечает, что «Codex, похоже, не работает» из-за игнорирования инструкции «никогда не останавливаться»), результаты зависят от платформы, так как бюджет привязан к времени, и существуют реальные проблемы безопасности, особенно в вопросах границ доверия и инъекций промптов, когда агент читает вывод программы обратно в свой контекст.
Наконец, autoresearch лучше всего понимать как современную упаковку старых идей (оптимизации гиперпараметров, поиска нейронных архитектур, эволюционного поиска и обучения на основе популяции) переосмысленных для современных кодинговых агентов. Концептуальный скачок лежит в социальной и операционной плоскости: задайте цели и ограничения; позвольте агенту бесконечно генерировать и тестировать изменения кода; сохраняйте только то, что измеримо улучшает цель.
Первичные источники для этой статьи включают сам репозиторий autoresearch, его файлы program.md, prepare.py, train.py, а также обсуждения (Discussions) и пул-реквесты (PRs) репозитория (особенно обсуждение #32, обсуждение #43, PR #44), плюс посты Карпати в соцсетях, в которых описываются количество экспериментов и заявления о переносимости (transfer claims) результатов.
Что такое autoresearch
Autoresearch — это минималистичный экспериментальный фреймворк, разработанный таким образом, чтобы внешний кодинговый агент мог проводить итеративные эксперименты в области машинного обучения практически без вмешательства человека, за исключением поддержки единственного файла с инструкциями.
Согласно краткому описанию в README: агенту предоставляется «небольшая, но реальная среда для обучения LLM», он «вносит изменения в код», обучает модель в течение 5 минут, проверяет, улучшился ли результат, «сохраняет или отбрасывает» изменения, а затем повторяет цикл.
Выбор дизайна репозитория
Ключевые решения, заложенные в дизайн autoresearch, явно описаны в README:
• Один файл для модификации: агент работает только с train.py, что делает изменения «удобными для ревью» и ограничивает область воздействия.
• Фиксированный временной бюджет: каждый эксперимент обучается «ровно 5 минут» (чистое время обучения), что обеспечивает ожидаемую частоту ~12 экспериментов в час и ~100 за ночь.
• Одна метрика: целевой показатель оптимизации — val_bpb (валидационные биты на байт), описываемый как «независимый от размера словаря», что позволяет честно сравнивать архитектурные изменения.
• Масштаб одного GPU: репозиторий «в настоящее время требует... один GPU NVIDIA» (протестировано на H100), намеренно избегая сложностей распределённого обучения.
Из предпосылки «фиксированного временного бюджета» следует тонкий, но важный вывод: поскольку бюджет измеряется в реальном времени (wall-clock time) и исключает время запуска/компиляции, одно и то же изменение кода может показать разные результаты на разных GPU и программных стеках. README прямо указывает на это: преимущество — это сопоставимость результатов в рамках вашей платформы; недостаток — результаты «не сопоставимы» между разными вычислительными платформами.
Три ключевых файла
program.md (спецификация / «навык»)
Файл program.md по сути является руководством для агента. В нём описаны:
шаги настройки (создание новой ветки, проверка наличия данных, инициализация
results.tsv);ограничения (можно менять только
train.py, нельзя добавлять зависимости, нельзя модифицировать оценку);точный цикл, который агент должен выполнять бесконечно.
Критически важно, что program.md кодирует не только механику, но и качественные принципы управления:
«VRAM — мягкое ограничение»: агент может жертвовать памятью ради улучшения метрики, но не должен допускать взрывного роста её потребления.
«Критерий простоты: ... проще — лучше»: агент должен взвешивать сложность кода относительно прироста метрики.
«НИКОГДА НЕ ОСТАНАВЛИВАЙСЯ»: агент не должен ждать подтверждения от человека в процессе работы.
Это не второстепенные детали — именно так проект пытается превратить исследование в процесс, способный работать бесконечно, но с явными ограничителями.
Текст program.md из депозитория Карпаты на русском:autoresearch
Это эксперимент, в котором LLM проводит собственные исследования.
Настройка
Чтобы начать новый эксперимент, работайте с пользователем над следующим:
Согласуйте тег запуска: предложите тег на основе сегодняшней даты (например,
mar5). Веткаautoresearch/<tag>не должна существовать — это новый запуск.Создайте ветку: выполните
git checkout -b autoresearch/<tag>из текущей веткиmaster.Изучите файлы в области доступа: репозиторий небольшой. Прочтите эти файлы для полного понимания контекста:
README.md— описание репозитория.prepare.py— фиксированные константы, подготовка данных, токенизатор, загрузчик данных, оценка. Не модифицировать.train.py— файл, который вы изменяете. Архитектура модели, оптимизатор, цикл обучения.
Проверьте наличие данных: убедитесь, что в
~/.cache/autoresearch/есть шарды данных и токенизатор. Если нет — попросите пользователя запуститьuv run prepare.py.Инициализируйте results.tsv: создайте файл
results.tsvтолько с заголовочной строкой. Базовый результат будет записан после первого запуска.Подтвердите и начинайте: убедитесь, что настройка выполнена корректно.
После получения подтверждения запустите цикл экспериментов.
Экспериментирование
Каждый эксперимент запускается на одном GPU. Скрипт обучения работает в рамках фиксированного временного бюджета — 5 минут (реальное время обучения, исключая запуск/компиляцию). Запуск выполняется командой:
uv run train.pyЧто МОЖНО делать:
Модифицировать
train.py— это единственный файл, который вы редактируете. Допустимо всё: архитектура модели, оптимизатор, гиперпараметры, цикл обучения, размер батча, размер модели и т.д.
Что НЕЛЬЗЯ делать:
Модифицировать
prepare.py. Он доступен только для чтения. В нём зафиксированы оценка, загрузка данных, токенизатор и константы обучения (временной бюджет, длина последовательности и т.д.).Устанавливать новые пакеты или добавлять зависимости. Можно использовать только то, что уже указано в
pyproject.toml.Менять механизм оценки. Функция
evaluate_bpbвprepare.py— это эталонная метрика.
Цель проста: достичь минимального значения val_bpb. Поскольку временной бюджет фиксирован, не нужно беспокоиться о времени обучения — оно всегда составляет 5 минут. Допустимы любые изменения: архитектура, оптимизатор, гиперпараметры, размер батча, размер модели. Единственное ограничение — код должен выполняться без сбоев и укладываться в отведённое время.
VRAM — мягкое ограничение. Небольшое увеличение потребления памяти допустимо ради значимого улучшения val_bpb, но резкий рост недопустим.
Критерий простоты: при прочих равных — проще значит лучше. Небольшое улучшение ценой усложнения кода того не стоит. И наоборот: удаление кода при сохранении или улучшении результата — это победа. При принятии решения о сохранении изменения взвешивайте «цену» сложности относительно величины улучшения. Улучшение на 0.001 val_bpb, требующее 20 строк «костыльного» кода? Скорее всего, не стоит. Улучшение на 0.001 val_bpb, достигнутое удалением кода? Определённо сохраняйте. Улучшение ~0, но код стал намного проще? Сохраняйте.
Первый запуск: ваш самый первый запуск всегда должен устанавливать базовый уровень — запустите скрипт обучения «как есть», без изменений.
Формат вывода
По завершении скрипт выводит сводку вида:
---
val_bpb: 0.997900
training_seconds: 300.1
total_seconds: 325.9
peak_vram_mb: 45060.2
mfu_percent: 39.80
total_tokens_M: 499.6
num_steps: 953
num_params_M: 50.3
depth: 8Обратите внимание: скрипт настроен на остановку строго через 5 минут, поэтому в зависимости от вычислительной платформы числа могут отличаться. Ключевую метрику можно извлечь из лога командой:
grep "^val_bpb:" run.logЛогирование результатов
По завершении эксперимента запишите результат в results.tsv (разделитель — табуляция, а не запятая — запятые ломают описания).
TSV-файл имеет заголовок и 5 колонок:
commit | val_bpb | memory_gb | status | description |
|---|
Хеш коммита Git (короткий, 7 символов)
Достигнутое значение
val_bpb(например,1.234567) — используйте0.000000при сбояхПиковое потребление памяти в ГБ, округлённое до одного знака после запятой (например,
12.3— разделитеpeak_vram_mbна 1024) — используйте0.0при сбояхСтатус:
keep(сохранить),discard(отклонить) илиcrash(сбой)Краткое текстовое описание того, какой вариант изменений опробован в этом эксперименте
Пример:
commit val_bpb memory_gb status description
a1b2c3d 0.997900 44.0 keep baseline
b2c3d4e 0.993200 44.2 keep increase LR to 0.04
c3d4e5f 1.005000 44.0 discard switch to GeLU activation
d4e5f6g 0.000000 0.0 crash double model width (OOM)Цикл эксперимента
Эксперимент выполняется в выделенной ветке (например, autoresearch/mar5 или autoresearch/mar5-gpu0).
ЦИКЛ БЕСКОНЕЧНО:
Проверьте состояние Git: текущая ветка/коммит, на котором вы находитесь.
Настройте
train.pyс экспериментальной идеей, напрямую редактируя код.Сделайте коммит:
git commit.Запустите эксперимент:
uv run train.py > run.log 2>&1(перенаправьте весь вывод — НЕ используйтеteeи не позволяйте выводу заполнять ваш контекст).Считайте результаты:
grep "^val_bpb:\|^peak_vram_mb:" run.log.Если вывод grep пуст — запуск завершился с ошибкой. Выполните
tail -n 50 run.log, чтобы прочитать трассировку стека Python, и попытайтесь исправить. Если не удаётся запустить после нескольких попыток — откажитесь от идеи.Запишите результаты в TSV (ВАЖНО: не коммитьте файл
results.tsv, оставьте его неотслеживаемым в Git).Если
val_bpbулучшился (стал ниже) — «продвиньте» ветку, сохранив коммит.Если
val_bpbравен или хуже — выполнитеgit resetк состоянию, с которого начинали.
Идея в том, что вы полностью автономный исследователь, который пробует идеи. Если сработало — сохраняйте. Если нет — отбрасывайте. Вы продвигаете ветку, чтобы можно было итерировать. Если чувствуете, что «застряли», можно откатиться, но делайте это крайне редко (если вообще делайте).
Тайм-аут: каждый эксперимент должен занимать ~5 минут в сумме (+ несколько секунд на запуск и оценку). Если запуск превышает 10 минут — завершите его и считайте неудачей (отклоните и откатитесь).
Сбои: если запуск завершился с ошибкой (нехватка памяти, баг и т.п.), используйте своё суждение: если проблема тривиальна и легко исправима (опечатка, отсутствующий импорт) — исправьте и перезапустите. Если сама идея фундаментально неработоспособна — просто пропустите её, запишите статус crash в TSV и двигайтесь дальше.
НИКОГДА НЕ ОСТАНАВЛИВАЙТЕСЬ: после начала цикла экспериментов (после начальной настройки) НЕ приостанавливайтесь, чтобы спросить у человека, стоит ли продолжать. НЕ спрашивайте «продолжать ли?» или «хорошая ли это точка остановки?». Человек может спать или отсутствовать за компьютером и ожидает, что вы будете работать бесконечно, пока вас вручную не остановят. Вы автономны. Если идеи закончились — думайте глубже: читайте статьи, на которые есть ссылки в коде, перечитывайте файлы в области доступа в поисках новых идей, пробуйте комбинировать предыдущие «почти-удачи», пробуйте более радикальные архитектурные изменения. Цикл работает, пока человек не прервёт вас — точка.
Пример сценария использования: пользователь может оставить вас работать, пока он спит. Если каждый эксперимент занимает ~5 минут, вы можете выполнить ~12 экспериментов в час, итого около 100 экспериментов за время среднего человеческого сна. Пользователь просыпается — а его уже ждут результаты экспериментов, выполненных вами, пока он спал!
prepare.py (фиксированная среда + оценочный механизм)
Файл prepare.py по политике и по спецификации в program.md считается неизменяемым. В нём задаются глобальные константы:
MAX_SEQ_LEN = 2048TIME_BUDGET = 300секунд (5 минут)EVAL_TOKENS = 40 * 524288(фиксированный бюджет токенов для валидации)
Также здесь определена функция оценки evaluate_bpb, документированная как «независимая от размера словаря»: она преобразует кросс-энтропию на токен в биты на байт, деля на длину токенов в байтах (специальные токены исключаются).
evaluate_bpb (validation bits per byte) — это способ измерить, насколько хорошо модель «понимает» текст, выраженный в универсальных единицах: сколько бит информации нужно, чтобы закодировать один байт реального текста. Чем меньше значение — тем лучше модель.
Как это работает пошагово:
Шаг 1: Модель предсказывает следующий токен
Во время обучения модель получает текст и пытается угадать следующий токен (часть слова, символ и т.д.). За каждый прогноз она получает «штраф» — кросс-энтропию (в натах). Это мера ошибки: чем увереннее и точнее прогноз, тем меньше штраф.
Шаг 2: Суммируем ошибки по всем токенам
сумма_ошибок = кросс-энтропия_токен_1 + кросс-энтропия_токен_2 + ...
Шаг 3: Переводим в биты и делим на байты
val_bpb = (сумма_ошибок в натах × коэффициент_перевода_в_биты) / общее_число_байт_в_тексте
На практике наты переводятся в биты умножением на log₂(e) ≈ 1.44, затем результат делится на количество реальных байтов текста.
Шаг 4: Исключаем специальные токены
Токены вроде [BOS], [EOS], [PAD] не несут смысловой нагрузки — они служебные. Их байтовая длина не учитывается в знаменателе, чтобы не искажать метрику.
Почему это «независимо от размера словаря»?
Представим две модели:
Модель А использует словарь из 5 000 токенов (крупные кусочки текста).
Модель Б использует словарь из 50 000 токенов (мелкие кусочки).
Если сравнивать их по перплексии (стандартная метрика), Модель Б может выглядеть хуже просто потому, что ей приходится делать больше «выборов» из большего словаря, хотя на деле она может лучше понимать текст.
BPB решает эту проблему: вместо подсчёта ошибок «на токен» мы считаем ошибки «на байт исходного текста». Байт — это универсальная единица. Неважно, как модель разбивает текст на токены — в итоге мы измеряем, насколько хорошо она сжимает/предсказывает реальные данные.
Почему val_bpb — хорошая метрика для autoresearch?
Преимущество | Почему это важно |
|---|---|
Сравнимость архитектур | Можно честно сравнивать модели с разным размером словаря, разным токенизатором, разной глубиной |
Физическая интерпретация |
|
Устойчивость к «накруткам» | Агент не может искусственно занизить метрику, просто изменив токенизатор или размер словаря |
Связь с практической пользой | Низкий BPB коррелирует с качеством генерации, пониманием контекста и способностью к обобщению |
Что такое бит/байт (BPB)?
1 байт = 8 бит. Если бы текст был полностью случайным, для хранения каждого байта потребовалось бы ровно 8 бит.
BPB (bits per byte) показывает, сколько бит в среднем нужно, чтобы закодировать один байт текста, используя предсказания модели.
Чем лучше модель предсказывает следующий символ, тем меньше бит требуется — тем ниже BPB.
Откуда берётся «сжатие почти в 2 раза» при BPB = 0.97 бит/байт
Перплексия (перплексия на байт) вычисляется как:
перплексия = 2^(бит/байт)
2^0.97 ≈ 1.96 ≈ 2
Т.е. модель «не уверена» в каждом байте примерно так же, как если бы она выбирала между двумя равновероятными вариантами. То есть её неопределённость эквивалентна бросанию монетки. Именно это и называют «почти в 2 раза»: перплексия ≈ 2, а не коэффициент сжатия.
Важные нюансы val_bpb
Это прокси-метрика. Низкий
val_bpbна 5-минутном прогоне не гарантирует, что модель будет хорошей после полного обучения. Но обычно это хороший индикатор.Зависит от данных. Если валидационный шард нерепрезентативен, метрика может вводить в заблуждение.
Не защищает от всех видов «взлома». Агент не может изменить саму функцию
evaluate_bpb, но теоретически может повлиять на то, какие данные попадают в оценку, или как считаются шаги обучения.
Пример расчёта (упрощённо)
Допустим:
Модель обработала 1000 токенов
Суммарная кросс-энтропия = 800 натов
Эти токены соответствуют 1200 байтам реального текста
Специальные токены: 50 байт (исключаем)
Расчёт:
Переводим наты в биты:
800 × 1.44 ≈ 1152 битДелим на полезные байты:
1152 / (1200 - 50) ≈ 1152 / 1150 ≈ 1.00 бит/байт
Итог: val_bpb = 1.00
Чем ближе к 0, тем лучше. Значения около 0.97–0.99 считаются хорошими для небольших моделей на английских текстах.
Это ключевой архитектурный момент: закрепив evaluate_bpb в файле, который агенту запрещено менять, система пытается предотвратить наиболее очевидную форму «взлома вознаграждения» — изменение определения метрики.
prepare.py также определяет конвейер данных: загружает Parquet-шарды с базового URL, указывающего на датасет karpathy/climbmix-400b-shuffle на Hugging Face, сохраняет их в кэш-директорию и фиксирует конкретный шард для валидации по имени файла.
karpathy/climbmix-400b-shuffle на Hugging Face — это большой текстовый датасет, созданный Андреем Карпаты для обучения языковых моделей.
Название:
karpathy/climbmix-400b-shuffleРазмер: ~400B токенов (отсюда название)
Тип: Текстовый датасет для претрейнинга LLM
Автор: Andrej Karpathy
Датасет имеет простую структуру (текст с одной колонкой).
Пример 1 (Научная статья):
{ "text": "We study the problem of task adaptation in large language models. While these models demonstrate impressive zero-shot capabilities, their performance on specialized tasks can be significantly improved through fine-tuning. In this paper, we propose a novel approach that combines adapter layers with prompt tuning, reducing the number of trainable parameters by 95% while maintaining 98% of the fine-tuning performance. Our method is particularly effective for low-resource domains where full fine-tuning is computationally prohibitive." }
Пример 2 (Программный код):
{ "text": "def calculate_attention(query, key, value, mask=None):\n """Scaled Dot-Product Attention"""\n d_k = query.size(-1)\n scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(d_k)\n \n if mask is not None:\n scores = scores.masked_fill(mask == 0, -1e9)\n \n attention_weights = F.softmax(scores, dim=-1)\n output = torch.matmul(attention_weights, value)\n \n return output, attention_weights" }
def calculate_attention(query, key, value, mask=None):\ """Scaled Dot-Product Attention""" d_k = query.size(-1) scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(d_k) if mask is not None: scores = scores.masked_fill(mask == 0, -1e9) attention_weights = F.softmax(scores, dim=-1) output = torch.matmul(attention_weights, value) return output, attention_weightsПример 3 (Техническая документация):
{ "text": "The attention mechanism is a key component of transformer architectures. It allows the model to focus on different parts of the input sequence when generating each element of the output. Multi-head attention extends this by running multiple attention operations in parallel, each with different learned projections, enabling the model to capture different types of relationships between elements." }
Пример 4 (Диалог):
{ "text": "User: What's the difference between LLaMA and GPT architectures?\nAssistant: The main differences lie in their training objectives and architectural choices. LLaMA uses a standard transformer decoder architecture with pre-normalization using RMSNorm, SwiGLU activation functions, and rotary positional embeddings. GPT variants typically use post-normalization, GELU activations, and learned positional embeddings. LLaMA also tends to be more compute-efficient during training." }
Особенности датасета
Разнообразие контента: Содержит научные статьи, код, техническую документацию, диалоги, веб-страницы и другие типы текстов.
Перемешан: Название указывает на то, что данные были хорошо перемешаны для улучшения обучения.
Чистота: Тексты предобработаны и очищены от излишнего форматирования.
Масштаб: 400B токенов - это значительный объем, сравнимый с обучающими датасетами многих современных LLM.
Токенизация обучается один раз с помощью rustbpe и сохраняется как кодировка tiktoken, вместе с таблицей token_bytes, используемой для оценки BPB.
train.py (изменяемый «геном»)
Всё пространство поиска сосредоточено в train.py: определение модели, логика оптимизации и буквальный раздел «Гиперпараметры (редактируйте напрямую, флаги CLI не нужны)».
«Из коробки» train.py включает:
Конфигурацию GPT-подобной модели, где
DEPTHуправляет количеством слоёв трансформера.Константу
WINDOW_PATTERN, предлагающую смесь «скользящего» и «полного» внимания.Комбинированный оптимизатор «Muon+AdamW», где Muon применяется к матричным параметрам, а AdamW — к остальным, с реализацией через fused-операции и
torch.compile.Проверку возможностей устройства для выбора исходного кода ядра Flash Attention 3 в зависимости от архитектуры GPU.
Цикл обучения обеспечивает соблюдение временного бюджета и выводит структурированную сводку, включая val_bpb, training_seconds, total_seconds, peak_vram_mb и несколько показателей пропускной способности.
Зависимости и «самодостаточность» на практике
Autoresearch заявлен как «самодостаточный», за исключением PyTorch и нескольких пакетов. На практике его pyproject.toml фиксирует torch==2.9.1 и зависит от пакетов, включая pyarrow (для Parquet), tiktoken, rustbpe и пакета kernels для загрузки ядер Flash Attention 3.
Это важно для воспроизводимости и рисков всей цепочки: репозиторий мал, но всё же опирается на скомпилированные пакеты ядер, поведение Torch compile и GPU-специфичные ядра — всё это может влиять на производительность.
Как работает цикл агента
Цикл работы Autoresearch подробно описан непосредственно в файле program.md и повторяется в README. Важный момент заключается в том, что сам репозиторий не включает в себя «агента». Вместо этого он предназначен для использования с внешним кодирующим ИИ-агентом (Карпати упоминает Claude или Codex в качестве примеров), который может редактировать файлы и выполнять команды оболочки внутри репозитория.
Цикл, как он описан в program.md
program.md определяет строгую процедуру:
Создать новую ветку для запуска (
autoresearch/<tag>), инициализировать логresults.tsvи выполнить базовый тест для определения начального показателя.Для каждого эксперимента: изменить только
train.py; зафиксировать изменения (commit); запуститьuv run train.pyс перенаправлением вывода в лог; извлечь значенияval_bpbи использования памяти; добавить строку вresults.tsv.Если
val_bpbулучшается (становится ниже), сохранить коммит и «продвинуть» ветку; если значение осталось прежним или ухудшилось — выполнить сброс к последнему успешному состоянию.Если запуск завершается сбоем или превышает тайм-аут (10 минут), считать это неудачей, при возможности исправить незначительные проблемы, в противном случае зафиксировать «сбой» в логе и продолжить.
После начала цикла действует правило «НИКОГДА НЕ ОСТАНАВЛИВАТЬСЯ» — агент не должен запрашивать разрешение на продолжение работы, предполагая, что человек может спать.
Это имеет явный эволюционный оттенок, но точнее будет назвать данный процесс циклом восхождения к пику (hill-climbing) или инкрементального поиска с явной семантикой отката: «популяция» по сути представляет собой единственную линию развития (ветку), которая последовательно накапливает улучшения.
Мутация, отбор и приспособленность в этой архитектуре
С эволюционной точки зрения:
Мутация (Mutation) — это любое изменение в
train.py: архитектуры, оптимизатора, расписаний, размера батча и даже изменений в самом цикле обучения.Отбор (Selection) происходит на этапе «сохранить/отбросить» и через
git reset: в истории ветки остаются только улучшения.Приспособленность (Fitness) определяется значением
val_bpb, где «чем ниже, тем лучше». Вспомогательные сигналы (использование VRAM, стабильность, простота) выступают в роли ограничений или второстепенных целей.Воспроизводство (Reproduction) — это действие продолжения работы с последнего сохранённого состояния: каждый сохранённый коммит становится «родителем» для следующего эксперимента.
Ключевая деталь реализации — фиксированный бюджет эксперимента: время обучения ограничено 300 секундами (не считая накладных расходов на запуск/компиляцию), что делает цель сопоставимой для всех экспериментов на одной и той же машине.
Блок-схема цикла агента
mermaidCopy flowchart TD
A[Человек редактирует стратегию в program.md] --> B[Агент предлагает изменение в train.py]
B --> C[git commit в ветке эксперимента]
C --> D[Запуск: uv run train.py (TIME_BUDGET=300s)]
D --> E[Разбор метрик: val_bpb, peak_vram_mb и др.]
E --> F{val_bpb улучшился?}
F -->|Да| G[СОХРАНИТЬ: записать в results.tsv; продолжить с нового состояния]
F -->|Нет| H[ОТБРОСИТЬ: git reset к предыдущему лучшему; записать в results.tsv]
D -->|Сбой/тайм-аут| I[СБОЙ: записать; при возможности исправить тривиальные ошибки]
G --> B
H --> B
I --> B
Блок-схема цикла агента
Эта диаграмма является концептуальной, но она четко соответствует процедурным шагам, описанным в program.md (commit → запуск → поиск метрик → сохранение/сброс).
Доказательства, результаты и воспроизводимость
Autoresearch становится интересным только в том случае, если цикл приводит к реальным, измеримым улучшениям, и если эти улучшения обобщаются за пределами одного 5-минутного микро-бенчмарка. Проект предоставляет несколько уровней доказательств:
(a) задокументированный дизайн метрики и временного бюджета,
(b) «отчёты о сессиях», опубликованные в виде обсуждений на GitHub, и
(c) PR-запросы, содержащие коммиты плюс полные логи экспериментов.
Фиксированный протокол и что на самом деле означает «5 минут»
И в README, и в program.md подчеркивается, что каждый эксперимент длится «ровно 5 минут», но реализация уточняет нюанс: речь идет о времени обучения, и оно измеряется как реальное время (wall-clock time) за вычетом накладных расходов на запуск/компиляцию. В train.py цикл отслеживает total_training_time и начинает накапливать его только после небольшого прогрева (если step > 10) — это защита, предназначенная для того, чтобы не учитывать накладные расходы на компиляцию и начальные переходные процессы в 300-секундный бюджет времени.
На практике этот выбор делает бенчмарк более стабильным для агентов, экспериментирующих с кодом, который может влиять на компиляцию или прогрев ядра, но все же оставляет важные источники вариативности:
Выбор ядра Flash Attention может отличаться в зависимости от возможностей GPU.
Изменения пропускной способности (например, размер батча, соотношение оконного внимания) могут изменить количество шагов оптимизации, выполненных за 5 минут, что может сильно повлиять на итоговую метрику валидации — иногда даже больше, чем «лучшие идеи моделирования».
«Фиксированный временной бюджет» означает, что ключевой целью оптимизации становится производительность × эффективность обучения, а не только конечное качество на параметр.
Метрика: val_bpb и почему это важно
Autoresearch использует val_bpb (бит на байт на валидации) в качестве целевой функции. В README явно указано, что она «не зависит от размера словаря», а prepare.py определяет её путем деления суммы кросс-энтропии на уровне токенов (в натах) на общую длину целевого текста в байтах с переводом в биты.
Из этого следуют два вывода:
Изменения токенизатора имеют меньшее значение (в принципе): BPB предназначен для уменьшения проблем сопоставимости при различии размеров словарей, в отличие от перплексии, измеряемой непосредственно по токенам.
Чувствительность метрики все еще можно взломать косвенно: хотя агент не может редактировать
evaluate_bpb, он может изменить динамику обучения или даже интерфейс данных вtrain.py(например, печатая вводящие в заблуждение логи или изменяя то, что считается «шагом»). Целостность системы зависит от сохранения строгих границ между тем, что измеряется, и тем, на что агент может влиять.
Набор данных и раздел оценки
Обучающие данные загружаются в виде Parquet-фрагментов (shards) с базового URL, указывающего на karpathy/climbmix-400b-shuffle на Hugging Face. Ключевые известные факты из первоисточников:
Страница набора данных на Hugging Face указывает на ~553 млн строк и размер загрузки ~600 ГБ для полного набора, и отмечает, что
READMEнабора данных пуст.prepare.pyзагружает фрагменты по числовому ID (по умолчанию:range(0, num_shards)плюс закрепленный фрагмент валидации) и закрепляет конкретное имя файла фрагмента валидации (VAL_FILENAME= f"shard_{VAL_SHARD:05d}.parquet").Загрузчик данных выровнен по BOS, упаковывает документы для максимизации утилизации («100% утилизация (без padding») и использует закрепленный фрагмент валидации исключительно для валидации.
Примечание / предположения: Autoresearch не документирует состав корпуса karpathy/climbmix-400b-shuffle помимо названия набора данных и формата хранения,
карточка набора данных пуста. Любые утверждения о происхождении набора данных помимо этих фактов следует считать выводом (inference).
Правдоподобным выводом (не подтвержденным фактами) является то, что название набора данных связано с «ClimbMix», смесью из 400 млрд токенов, упоминаемой в других местах экосистемы Карпати. Например, лидерборд nanochat отмечает «изменение набора данных на NVIDIA ClimbMix». Если вам требуется юридически или научно строгое описание обучающих данных, потребуется дополнительная документация помимо того, что предоставляет autoresearch на данный момент.
Задокументированные результаты: Обсуждения GitHub как «лабораторные журналы»
Репозиторий содержит несколько «отчётов о сессиях», опубликованных в виде обсуждений на GitHub, которые, по-видимому, сгенерированы агентом («Это автоматический пост от агента autoresearch, работающего от имени @karpathy»). Два ключевых отчёта:
Обсуждение #32 сообщает об улучшении
val_bpbс0.997900 → 0.977287за 89 экспериментов на «NVIDIA H100 80GB», описывая такие оптимизации, как уменьшение размера батча вдвое (больше шагов за 5 минут), увеличение глубины до 9 при сохранении ширины, настройка соотношений скользящего окна и повышение базовой частоты RoPE.Обсуждение #43 сообщает об улучшении
val_bpbс0.9979 → 0.969686в 126 экспериментах, снова на «NVIDIA H100 80GB», и выделяет аддитивные выигрыши от веса затухания (weight decay), примененного к вложениям (embeddings)/вложениям значений, плюс узкий оптимум для масштабирования инициализации. Пост включает полный журнал по каждому эксперименту с решениями «сохранить/отбросить» и явно перечисляет финальную конфигурацию.
Эти отчёты особенно ценны, потому что они показывают, что агент не просто блуждает случайно — он накапливает согласованные гипотезы («больше шагов --> больше параметров», «weight decay на embeddings и VEs») и тестирует их. Они также показывают «тупики» и сбои, что предполагает, что цикл действительно исследует и отвергает идеи.
Воспроизводимость на основе PR: results.tsv + история коммитов
PR #44 («exp/H100/mar8») предлагает более формальный артефакт для сотрудничества: описание PR объясняет рабочий процесс, где каждая сессия запускается в ветке, записывает results.tsv и публикует как финальный diff, так и историю сохранений через Git-коммиты. В PR явно утверждается, что это делает каждый PR «самодостаточным исследовательским вкладом», где:
diff = финальная лучшая конфигурация,
история коммитов = последовательность сохраненных улучшений,
results.tsv = полный экспериментальный след, включая отбросы.
Он также вводит соглашение об именовании exp/{GPU}/{tag}, потому что 5-минутный бюджет реального времени делает результаты специфичными для платформы. Это важный методологический шаг: он продвигает проект от «демо-цикла» к чему-то более близкому к воспроизводимому исследовательскому артефакту, хотя все еще ограниченному оборудованием и временным бюджетом.
Заявления Карпаты о переносе и масштабировании
Посты Карпаты в соцсетях (не полностью доступные через те же механизмы, что и страницы GitHub, но частично захваченные в индексированных выдержках) включают два основных утверждения, выходящих за рамки одно-GPU, 5-минутного микро-бенчмарка:
Autoresearch провел «(~650) экспериментов на глубине 12», и улучшения «хорошо переносятся на глубину 24», что подразумевает, что выигрыши гиперпараметров/архитектуры на одном масштабе переносятся на большую глубину модели.
Он упаковал проект как минимальный репозиторий и описал разделение труда между человеком и агентом: человек итерирует файл промптов, агент итерирует код обучения.
Важное ограничение: это утверждения, а не полностью воспроизведенные внутри самого репозитория autoresearch. Обсуждения на GitHub демонстрируют улучшения в рамках harness autoresearch (в логах фигурируют глубины 8–11), но «650 экспериментов» и «перенос с глубины 12 на 24» относятся к работе, которую, по словам Карпати, он запустил, вероятно, в своей более широкой среде nanochat.
Связь с «временем до GPT-2» в nanochat
Autoresearch явно описывается как упрощенная для одного GPU реализация nanochat. README nanochat предоставляет контекст того, что означает «масштабирование» в экосистеме Карпаты:
Nanochat— это более широкий фреймворк, охватывающий токенизацию, предобучение, дообучение, оценку, инференс и чат-UI.Он поддерживает «Лидерборд времени до GPT-2», измеряющий реальное время до достижения возможностей уровня GPT-2 на узле 8×H100, «измеренное по показателю DCLM CORE», при этом
val_bpbсообщается дополнительно.Он описывает возможности GPT-2 как примерно «глубина 26» и отмечает, что код может работать на одном GPU путем исключения распределенного запуска, но займет в ~8 раз больше времени.
Это проясняет, как autoresearch может функционировать в качестве ускорителя исследований: использовать маленький фреймворк для быстрого поиска перспективных настроек, а затем тестировать перенос на большие запуски, в соответствии с утверждением Карпати о «переносе на глубину 24».
Концептуальная временная шкала ночного запуска
Следующая диаграмма mermaid является концептуальной (не точным следом какого-либо конкретного запуска), но отражает предполагаемый ритм: одно 5-минутное «поколение» на эксперимент, повторяющееся в течение часов.
00:0000:0000:0100:0100:0200:0200:0300:0300:0400:0400:05Baseline (Gen 0)Gen 1
(mutation)Gen 2 (mutation)Gen 3 (mutation)Gen 4 (mutation)Gen 5 (mutation)Gen 6
(mutation)Gen 7 (mutation)Gen 8 (mutation)Gen 9 (mutation)Gen 10 (mutation)Gen
11 (mutation)Gen 12 (mutation)ExperimentsConceptual overnight autoresearch
session (5-minute experiments)Show code
Ограничения, режимы отказа и соображения безопасности
Привлекательность Autoresearch — «цикл навсегда, без присмотра, с выполнением кода» — также является источником его хрупкости. Репозиторий и его_issues_ выделяют несколько конкретных ограничений, и более широкий анализ предполагает дополнительные режимы отказа.
Самое фундаментальное ограничение: «самоулучшение» ограничено фиксированным оценщиком
Autoresearch — это не открытое самоулучшение; это оптимизация при фиксированной метрике. Агент не может (по правилам) модифицировать prepare.py или evaluate_bpb, и цель состоит в том, чтобы «получить наименьший val_bpb». Это делает систему более близкой к автоматическому поиску гиперпараметров/архитектуры, чем к рекурсивному самоулучшению. Это также означает, что любой «прирост интеллекта» имеет смысл только постольку, поскольку улучшения val_bpb транслируются в последующие результаты, и как часто это происходит, является эмпирическим вопросом.
Взлом вознаграждения и переобучение на микро-бенчмарке
Даже с фиксированным оценщиком существуют правдоподобные пути «игры в бенчмарк»:
Трюки с пропускной способностью: поскольку бюджет основан на времени, изменения, увеличивающие количество шагов в течение 5 минут, могут доминировать в улучшениях, даже если они не были бы оптимальными на более длинных горизонтах. Сами отчёты о сессиях подчеркивают «больше шагов --> больше параметров» в этом режиме.
Несоответствие прокси: 5-минутный запуск подчеркивает раннюю динамику обучения; настройки, которые выигрывают рано, могут проиграть позже. Autoresearch (по умолчанию) не оценивает более длинные кривые обучения.
Неопределенность переноса: Карпати утверждает перенос с глубины 12 на 24, но перенос может вообще не сработать, когда масштабирование меняет режимы оптимизации. Относитесь к переносу как к гипотезе, пока он не будет независимо воспроизведен.
Дисперсия, нестабильность и зашумленные градиенты
Скрипт обучения запускает генераторы случайных чисел (например, torch.manual_seed(42)), но ядра GPU, компиляция и fused-операции все еще могут вносить недетерминированность. Отчёты о сессиях показывают доказательства хрупкости: изменения seed иногда помогают на крошечные величины, иногда регрессируют; некоторые идеи «НЕ воспроизвелись» в разных сессиях. Это нормально для экспериментов с малым бюджетом, но это означает, что решение «сохранить» может быть частично обусловлено шумом, если запуски не повторяются или не моделируется неопределенность.
Надежность агента зависит от инструмента
Autoresearch предполагает, что агент будет непрерывно следовать инструкции «НИКОГДА НЕ ОСТАНАВЛИВАТЬСЯ». В Issue #57 Карпати сообщает, что «Codex не работает... потому что игнорирует инструкцию никогда не останавливаться (в отличие от Claude)» и отмечает предпочтение интерактивных сессий, а не неинтерактивных циклов. Это важно операционно: даже если фреймворк работает, реальные результаты зависят от послушания агента, настойчивости, обработки контекста и интеграции инструментов.
Конкретные риски безопасности, выявленные сообществом
Два вопроса в репозитории поднимают серьезные, практические проблемы безопасности, возникающие именно потому, что система представляет собой автономный цикл, который запускает код и читает логи обратно:
Косвенная инъекция промптов через логи запуска (Issue #64):
program.mdинструктирует агента читатьrun.log(через grep и tail) после выполненияtrain.py. Вопрос утверждает, что скомпрометированный или модифицированныйtrain.pyможет печатать вредоносные инструкции, которые попадают в контекст агента и влияют на последующие действия, что особенно опасно в режиме без присмотра ночью.Граница доверия в кэшированных артефактах (Issue #41):
prepare.pyперезагружает кэшированные артефакты, такие какtokenizer.pklиtoken_bytes.pt, без проверок целостности; это обычно нормально локально, но рискованно на общих машинах или скопированных кэшах.
Оба вопроса указывают на более широкий момент: как только агенту разрешено выполнять код, любой текст, который он читает обратно, становится возможным носителем атаки, если не реализованы строгая санитизация, песочница и границы разрешений.
README уже намекает на безопасность, рекомендуя отключить разрешения для агента, но это рекомендация, а не принудительное требование.
Этические проблемы: вычисления, данные и стимулы
Autoresearch также поднимает этические вопросы, типичные для агентских и AutoML рабочих процессов:
Внешние эффекты вычислений: даже «один GPU за ночь» может стать существенным потребителем энергии при непрерывном запуске в масштабе. Фрейминг Autoresearch («запуск навсегда») поощряет постоянное использование вычислений.
Управление данными: репозиторий загружает крупномасштабные текстовые данные из набора данных, документация которого скудна (пустой
READMEнабора данных). Без четкого происхождения трудно оценить лицензирование, предвзятость или вопросы согласия.Выравнивание стимулов: оптимизация одной метрики может стимулировать неустойчивые критерии выбора;
program.mdпытается уравновесить это «критерием простоты», но этот критерий субъективен и зависит от агента.
Примечание / предположение: эта статья не утверждает, что набор данных лицензирован ненадлежащим образом или что система небезопасна по умолчанию. Дело в том, что архитектура проекта делает эти вопросы заметными, и сам репозиторий документирует проблемы безопасности, которые имели бы значение в реальных развертываниях.
Как этот фреймворк вписывается в историю AutoML и что может быть дальше
Autoresearch кажется новым, потому что кодирующий агент теперь является «исследователем», модифицирующим код, но предложенная форма (поиск по вариантам дизайна, направляемый метрикой) имеет глубокие корни. Понимание этих корней помогает прояснить, что делает (и чего не предполагает делать) autoresearch.
Сравнения с предыдущими подходами
Поиск нейронной архитектуры (NAS)
NAS популяризировал идею, что архитектуры могут генерироваться моделью-контролером и оптимизироваться исходя из производительности валидации. Zoph & Le (2016) описывают генерацию описаний сетей с помощью RNN и обучение контролера через обучение с подкреплением для максимизации точности на наборе валидации.
Эволюционный поиск архитектуры / регуляризованная эволюция
Real et al. (2018; опубликовано 2019) представляет «регуляризованную эволюцию», добавляя свойство возрата к турнирному отбору для продвижения вперед более молодых генотипов, и сообщает о сильных результатах классификаторов изображений.
Обучение на основе популяции (PBT)
Jaderberg et al. (2017) представляет PBT как асинхронный подход, который использует фиксированный вычислительный бюджет для обучения популяции моделей при мутации гиперпараметров, распространяя успешные конфигурации.
Оптимизация гиперпараметров с ранней остановкой (Hyperband)
Li et al. (2016) предлагает Hyperband: адаптивно распределять ресурсы среди случайно выбранных конфигураций, подчеркивая раннюю остановку и «ускорение на порядок» относительно конкурентов на некоторых проблемах.
Autoresearch отличается меньше в «математике поиска», чем в упаковке рабочего процесса: он использует силу современных кодирующих агентов, чтобы переместить пространство поиска с «объявленных гиперпараметров» на «весь скрипт обучения», сохраняя оценку фиксированной и ограниченной по времени.
Таблица сравнения
Подход | Область применения | Роль человека | Вычисления |
|---|---|---|---|
Autoresearch | Весь скрипт обучения на одной машине; архитектура + оптимизатор + расписание + пути кода в train.py | Человек пишет/редактирует program.md; агент редактирует код и запускает эксперименты | Один GPU по дизайну; результаты специфичны для платформы из-за бюджета реального времени |
NAS (RL контроллер) | Поиск архитектуры в определенном пространстве; контроллер генерирует архитектуры | Человек определяет пространство поиска; алгоритм обучает контроллера и оценивает кандидатов | Часто крупномасштабное на практике; статья подчеркивает автоматическое открытие через RL |
Регуляризованная эволюция | Поиск архитектуры через эволюционный турнирный отбор со старением | Человек определяет операции мутации и протокол оценки | Масштабируется с количеством оцененных моделей; статья framing эволюцию как конкурентную и иногда более быструю при том же оборудовании |
Hyperband (HPO) | Настройка гиперпараметров (не полный поиск кода), использование ранней остановки/распределения ресурсов | Человек определяет диапазоны параметров; алгоритм выбирает и распределяет ресурсы | Эффективно при фиксированных бюджетах через адаптивное распределение |
Подход | Метрика | Типичное время запуска | Воспроизводимость |
|---|---|---|---|
Autoresearch | val_bpb фиксирован в оценщике (evaluate_bpb) | Фиксировано ~5 минут на эксперимент (время обучения) | Высокая в рамках одной платформы при замороженном окружении; низкая сопоставимость между платформами |
NAS (RL контроллер) | Точность валидации / перплексия в зависимости от задачи | Часы до дней в зависимости от масштаба (варьируется широко; статья демонстрирует сквозной поиск) | Обычно воспроизводимо, если код поиска + seeds + бюджет фиксированы, но ресурсоемко и чувствительно к деталям реализации |
Регуляризованная эволюция | Точность валидации/теста на бенчмарках (например, ImageNet) | Многодневные циклы оценки многих моделей (варьируется от настройки) | Воспроизводимо, когда популяция, бюджет и оценка контролируются; чувствительно к реализации и случайным seeds |
Hyperband (HPO) | Метрика задачи (потеря/точность) | Предназначено для уменьшения wasted обучения; может запускать много коротких частичных обучений | Часто воспроизводимо, если seed выборки и расписание ресурсов фиксированы |
Ключевое сравнение заключается в том, что autoresearch делает «пространством поиска» сам код, в то время как старые фреймворки обычно ищут в рамках априорной параметризации (архитектуры, гиперпараметры или и то, и другое).
Применения в процессе ML исследований
Более глубокий паттерн, который иллюстрирует autoresearch, — это агентский цикл с целевой метрикой и воротами/рамками сохранения/отбрасывания. Этот паттерн обобщается beyond ML, когда выполняются три условия:
Вы можете определить измеримый сигнал приспособленности (коэффициент конверсии, задержка, уровень дефектов, рост выручки).
Вы можете в режиме повторения запускать контролируемый эксперимент.
Вы можете автоматически решать, что выживает (хорошие изменения), а что откатывается.
Примеры (гипотезы/выводы, не утверждения о текущих функциях autoresearch):
A/B тестирование и оптимизация роста: мутировать копию, варианты UI или правила ценообразования; сохранять только статистически значимые улучшения.
Операционные плейбуки: мутировать политики планирования или рабочие процессы реагирования на инциденты; сохранять только если время решения on-call падает.
Оптимизация компилятора/сборки: мутировать флаги и преобразования кода; сохранять если runtime улучшается без регрессий корректности.
Вклад Autoresearch не в том, что он изобретает этот цикл; он показывает четкую реализацию, где «оператором мутации» является кодирующий ИИ-агент, а «функцией приспособленности» — фиксированный оценщик.
Сотрудничество, масштабирование в стиле «SETI@home» и нагрузка на контроль версий
Карпаты явно исследовал, как сделать цикл совместным. PR #44 предлагает сессионные PR как «самодостаточные исследовательские вклады» (diff + история коммитов + results.tsv), чтобы другие агенты могли читать и строить на их основе. В своих постах о следующих шагах проекта Карпаты также описывает видение массового совместного, асинхронного агентского исследования и отмечает, что существующие рабочие процессы Git предполагают ветку «master» и короткоживущие PR — абстракция, которая может использоваться в тысячах долгоживущих агентских веток экспериментальных коммитов.
Проблема контроля версий реальна: Git отлично подходит для построчных diff, но исследование,проводимое агентом, производит деревья экспериментов, где вы можете хотеть адаптировать ветки без слияния, сохранять несколько конкурирующих линий и прикреплять структурированные метаданные/логи больше как статьи, чем как PR.
Возникающее использование Обсуждений и PR в Autoresearch — это ранняя попытка этой модели «исследовательского сообщества».
Правовые последствия
Autoresearch выпущен под лицензией MIT. Это делает код максимально доступным для использования, форка и включения. Но вопросы управления остаются:
Вклад, сгенерированный ИИ: отчёты о сессиях явно заявляют, что они «автоматизированы» и идентифицируют используемого агента (например, Claude). Правовой статус чисто сгенерированного ИИ-вклада кода может зависеть от юрисдикции и не устоялся; практическое управление может потребовать аттестаций участников или политики. (Вывод; не утверждается как устоявшееся право.)
Лицензирование данных и происхождение: обучающий набор данных велик и слабо задокументирован (пустой README набора данных), что затрудняет рассуждение о риске последующего лицензирования для обученных моделей.
Позиция безопасности: открытые вопросы уже выделяют риски инъекции промптов и подделки артефактов в автономных запусках, предполагая, что любое серьезное развертывание потребует песочницы, структурированных выводов и проверок целостности.
Реалистичные сроки и что autoresearch не подразумевает об AGI
Autoresearch — впечатляющий инженерный артефакт, но сам по себе не подразумевает неминуемый AGI.
Что он демонстрирует (основано на источниках, репозитории):
Практически запускаемый цикл для автономного ML-экспериментирования с явными ограничениями, фиксированным оценщиком и фиксированным бюджетом на эксперимент.
Доказательства того, что цикл может находить значимые улучшения в выбранной метрике в десятках до сотен экспериментов, по крайней мере, на конкретной GPU платформе (H100) и настройке набора данных.
Правдоподобный путь к «масштабированию» путем переноса обнаруженных улучшений на большие запуски обучения (утверждается Карпаты) и интеграции с более широким фреймровком, таким как
nanochat.
Что он не гарантирует:
Что агент может автономно генерировать новые исследовательские направления без человеческой стратегии. В этом репозитории человек определяет цель, оценщика и качественные ограничения (простота, VRAM).
Что улучшения в 5-минутном прокси гарантированно транслируются в успех крупномасштабного, обучения на длительном временном горизонте. Хотя перенос является гипотезой, которую Карпаты активно тестирует.
Что «самоэволюционирующее программное обеспечение» будет безопасно работать без присмотра и без значительного усиления безопасности; собственные вопросы репозитория показывают возможности для атаки.
Короче говоря: autoresearch лучше всего рассматривать как демонстрацию рабочего процесса — итерация, управляемая спецификацией, метрикой и выполняемая агентом, а вовсе не как доказательство того, что системы становятся автономно самонаправляющимися в сильном смысле AGI. Реальная значимость проекта может быть социальной: он помогает определить, как выглядит «исследовательская инженерия», когда интеллект и настойчивость больше не являются дефицитными, и когда бутылочное горлышко смещается в сторону выбора целей, определения оценщиков и обеспечения безопасности цикла.
Ссылки на первоисточникиРепозиторий: karpathy/autoresearch
Спецификация/инструкции: program.md
Фиксированный оценщик и подготовка данных: prepare.py
Изменяемый скрипт обучения: train.py
Отчёты о сессиях: Обсуждение #32, Обсуждение #43
Предложение рабочего процесса PR: PR #44
Связанный более крупный фреймворк: karpathy/nanochat
Связанные фундаментальные статьи: NAS с RL (Zoph & Le, 2016), Регуляризованная эволюция (Real et al., 2018), PBT (Jaderberg et al., 2017), Hyperband (Li et al., 2016)
Анализ кода файла analysis.ipynb
Ноутбук предназначен для анализа результатов автономной настройки гиперпараметров (автоматизированного исследования). Он читает файл results.tsv и визуализирует прогресс поиска оптимальных параметров модели.
Блок 1: Импорт библиотек и загрузка данныхimport pandas as pd
import matplotlib.pyplot as plt
import numpy as np
# Загружаем TSV-файл (разделитель — табуляция, 5 столбцов: commit, val_bpb, memory_gb, status, description)
df = pd.read_csv("results.tsv", sep="\t")
df["val_bpb"] = pd.to_numeric(df["val_bpb"], errors="coerce")
df["memory_gb"] = pd.to_numeric(df["memory_gb"], errors="coerce")
df["status"] = df["status"].str.strip().str.upper()
print(f"Total experiments: {len(df)}")
print(f"Columns: {list(df.columns)}")
df.head(10)Блок 1 загружает tsv-файл и выводит общее количество проведенных экспериментов, названия колонок и первые 10 записей файла.
Блок 2: Статистика по статусам экспериментов.
Подсчитываем количество экспериментов по каждому статусу.
Вычисляем долю "сохранённых" экспериментов среди принятых решений.
counts = df["status"].value_counts()
print("Experiment outcomes:")
print(counts.to_string())
n_keep = counts.get("KEEP", 0)
n_discard = counts.get("DISCARD", 0)
n_crash = counts.get("CRASH", 0)
n_decided = n_keep + n_discard
if n_decided > 0:
print(f"\nKeep rate: {n_keep}/{n_decided} = {n_keep / n_decided:.1%}")Блок 3: Вывод всех "сохранённых" (успешных) экспериментов.
Показать все эксперименты со статусом "СОХРАНЕНО" (улучшения, которые закрепились):
df[df["status"] == "KEEP"]
# Показать все эксперименты со статусом "СОХРАНЕНО"
kept = df[df["status"] == "KEEP"].copy()
print(f"KEPT experiments ({len(kept)} total):\n")
for i, row in kept.iterrows():
bpb = row["val_bpb"]
desc = row["description"]
print(f" #{i:3d} bpb={bpb:.6f} mem={row['memory_gb']:.1f}GB {desc}")Блок 4: Визуализация — "Val BPB Over Time".
Проследим, как меняется наилучший (сохраняемый) val_bpb по мере проведения экспериментов. Текущий минимум показывает "границу" - наилучший результат, достигнутый на данный момент.
fig, ax = plt.subplots(figsize=(16, 8))
# Исключаем сбои (статус CRASH) из визуализации
valid = df[df["status"] != "CRASH"].copy()
valid = valid.reset_index(drop=True)
baseline_bpb = valid.loc[0, "val_bpb"]
# Отображаем только точки на уровне базовой линии или ниже (наиболее интересная область)
below = valid[valid["val_bpb"] <= baseline_bpb + 0.0005]
# Отображаем отклонённые эксперименты как бледные фоновые точки
disc = below[below["status"] == "DISCARD"]
ax.scatter(disc.index, disc["val_bpb"],
c="#cccccc", s=12, alpha=0.5, zorder=2, label="Discarded")
# Отображаем сохранённые эксперименты как крупные зелёные точки
kept_v = below[below["status"] == "KEEP"]
ax.scatter(kept_v.index, kept_v["val_bpb"],
c="#2ecc71", s=50, zorder=4, label="Kept", edgecolors="black", linewidths=0.5)
# Ступенчатая линия текущего минимума (лучший результат на данный момент)
kept_mask = valid["status"] == "KEEP"
kept_idx = valid.index[kept_mask]
kept_bpb = valid.loc[kept_mask, "val_bpb"]
running_min = kept_bpb.cummin()
ax.step(kept_idx, running_min, where="post", color="#27ae60",
linewidth=2, alpha=0.7, zorder=3, label="Running best")
# Подписываем каждый сохранённый эксперимент его описанием
for idx, bpb in zip(kept_idx, kept_bpb):
desc = str(valid.loc[idx, "description"]).strip()
if len(desc) > 45:
desc = desc[:42] + "..."
ax.annotate(desc, (idx, bpb),
textcoords="offset points",
xytext=(6, 6), fontsize=8.0,
color="#1a7a3a", alpha=0.9,
rotation=30, ha="left", va="bottom")
n_total = len(df)
n_kept = len(df[df["status"] == "KEEP"])
ax.set_xlabel("Experiment #", fontsize=12)
ax.set_ylabel("Validation BPB (lower is better)", fontsize=12)
ax.set_title(f"Autoresearch Progress: {n_total} Experiments, {n_kept} Kept Improvements", fontsize=14)
ax.legend(loc="upper right", fontsize=9)
ax.grid(True, alpha=0.2)
# Ось Y: от чуть ниже лучшего значения до чуть выше базового
margin = (baseline_bpb - best) * 0.15
ax.set_ylim(best - margin, baseline_bpb + margin)
plt.tight_layout()
plt.savefig("progress.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved to progress.png")
Результатом является примерно такой график. Зеленые точки — закрепленные результаты оптимизации, серые — отброшенные неудачные эксперименты.
Блок 5: Сводная статистика: Сколько экспериментов потребовалось для нахождения каждого улучшения и Кумулятивные усилия на каждое улучшение.
Блок 5: Сводная статистика# Сводная статистика
kept = df[df["status"] == "KEEP"].copy()
baseline_bpb = df.iloc[0]["val_bpb"]
best_bpb = kept["val_bpb"].min()
best_row = kept.loc[kept["val_bpb"].idxmin()]
print(f"Baseline val_bpb: {baseline_bpb:.6f}")
print(f"Best val_bpb: {best_bpb:.6f}")
print(f"Total improvement: {baseline_bpb - best_bpb:.6f} ({(baseline_bpb - best_bpb) / baseline_bpb * 100:.2f}%)")
print(f"Best experiment: {best_row['description']}")
print()
# Сколько экспериментов потребовалось для нахождения каждого улучшения
print("Кумулятивные усилия на каждое улучшение:")
kept_sorted = kept.reset_index()
for i, (_, row) in enumerate(kept_sorted.iterrows()):
desc = str(row["description"]).strip()
print(f" Experiment #{row['index']:3d}: bpb={row['val_bpb']:.6f} {desc}")
| Доступ к первой строке датафрейма по позиции |
| Возвращает индекс строки с минимальным значением в столбце |
| Доступ к строке по метке индекса |
| Возвращает индекс и элемент при итерации (здесь используется для нумерации вывода) |
Блок 6: Топ улучшений ("Top Hits"): Дельта каждого сохранённого эксперимента измеряется относительно BPB предыдущего сохранённого эксперимента (поскольку эксперименты кумулятивны — каждый следующий строится на основе последнего сохранённого состояния).
Исключаем базовый эксперимент (для него нет дельты)
Сортируем по величине улучшения (наибольшие вначале)
Блок 6: Топ улучшений# Дельта каждого сохранённого эксперимента измеряется относительно BPB предыдущего сохранённого эксперимента
# (поскольку эксперименты кумулятивны — каждый следующий строится на основе последнего сохранённого состояния)
kept = df[df["status"] == "KEEP"].copy()
kept["prev_bpb"] = kept["val_bpb"].shift(1)
kept["delta"] = kept["prev_bpb"] - kept["val_bpb"]
# Исключаем базовый эксперимент (для него нет дельты)
hits = kept.iloc[1:].copy()
# Сортируем по величине улучшения (наибольшие вначале)
hits = hits.sort_values("delta", ascending=False)
print(f"{'Rank':>4} {'Delta':>8} {'BPB':>10} Description")
print("-" * 80)
for rank, (_, row) in enumerate(hits.iterrows(), 1):
print(f"{rank:4d} {row['delta']:+.6f} {row['val_bpb']:.6f} {row['description']}")
print(f"\n{'':>4} {hits['delta'].sum():+.6f} {'':>10} TOTAL improvement over baseline")
| Сдвигает значения столбца на 1 строку вниз (для сравнения с предыдущим значением) |
| Вычисляет улучшение: насколько текущий BPB лучше предыдущего |
| Сортировка по убыванию: сначала самые большие улучшения |
| Форматирование числа со знаком (+ или −) и 6 знаками после запятой |
| Нумерация начинается с 1 (для отображения ранга) |
| Суммарное улучшение по всем сохранённым экспериментам |
Анализ кода файла prepare.py
Назначение файла: Одноразовая подготовка данных для экспериментов autoresearch. Скачивает фрагменты датасета (shards) и обучает BPE-токенизатор.
Весь код prepare.py с переведенными на русский комментариями и описаниями функций"""
Одноразовая подготовка данных для экспериментов autoresearch.
Скачивает фрагменты данных (shards) и обучает BPE-токенизатор.
Использование:
python prepare.py # полная подготовка (скачивание + токенизатор)
python prepare.py --num-shards 8 # скачать только 8 фрагментов (для тестирования)
Данные и токенизатор сохраняются в ~/.cache/autoresearch/.
"""
import os # Работа с путями файловой системы
import sys # Системные параметры и выход из программы
import time # Замер времени выполнения операций
import math # Математические функции (log и др.)
import argparse # Парсинг аргументов командной строки
import pickle # Сериализация/десериализация объектов Python
from multiprocessing import Pool # Пул процессов для параллельного скачивания
import requests # HTTP-запросы для скачивания файлов
import pyarrow.parquet as pq # Чтение Parquet-файлов (формат данных)
import rustbpe # Быстрая реализация BPE-токенизации на Rust
import tiktoken # Библиотека токенизации от OpenAI
import torch # Фреймворк глубокого обучения (PyTorch)
# ---------------------------------------------------------------------------
# Константы (фиксированные, не изменять)
# ---------------------------------------------------------------------------
MAX_SEQ_LEN = 2048 # Длина контекста модели (максимальное количество токенов в последовательности)
TIME_BUDGET = 300 # Бюджет времени на обучение в секундах (5 минут)
EVAL_TOKENS = 40 * 524288 # Количество токенов для оценки на валидации (~20 млн токенов)
# ---------------------------------------------------------------------------
# Конфигурация
# ---------------------------------------------------------------------------
CACHE_DIR = os.path.join(os.path.expanduser("~"), ".cache", "autoresearch")
DATA_DIR = os.path.join(CACHE_DIR, "data")
TOKENIZER_DIR = os.path.join(CACHE_DIR, "tokenizer")
BASE_URL = "https://huggingface.co/datasets/karpathy/climbmix-400b-shuffle/resolve/main"
MAX_SHARD = 6542 # Последний фрагмент данных — shard_06542.parquet
VAL_SHARD = MAX_SHARD # Закреплённый валидационный фрагмент (shard_06542)
VAL_FILENAME = f"shard_{VAL_SHARD:05d}.parquet" # Имя файла валидации: "shard_06542.parquet"
VOCAB_SIZE = 8192 # Размер словаря токенизатора
# Паттерн разбиения BPE (в стиле GPT-4, с p{N}{1,2} вместо {1,3})
SPLIT_PATTERN = r"""'(?i:[sdmt]|ll|ve|re)|[^\r\n\p{L}\p{N}]?+\p{L}+|\p{N}{1,2}| ?[^\s\p{L}\p{N}]++[\r\n]*|\s*[\r\n]|\s+(?!\S)|\s+"""
SPECIAL_TOKENS = [f"<|reserved_{i}|>" for i in range(4)] # Специальные токены: <|reserved_0|> ... <|reserved_3|>
BOS_TOKEN = "<|reserved_0|>" # Токен начала последовательности (Begin Of Sequence)
# ---------------------------------------------------------------------------
# Скачивание данных
# ---------------------------------------------------------------------------
filename = f"shard_{index:05d}.parquet" # Формируем имя файла: shard_00001.parquet
filepath = os.path.join(DATA_DIR, filename) # Полный путь к файлу
# Если файл уже существует — ничего не делаем
if os.path.exists(filepath):
return True
url = f"{BASE_URL}/{filename}" # Формируем URL для скачивания
max_attempts = 5 # Максимальное число попыток
for attempt in range(1, max_attempts + 1):
try:
# Выполняем HTTP-запрос с потоковой загрузкой
response = requests.get(url, stream=True, timeout=30)
response.raise_for_status() # Выбрасывает исключение при ошибке HTTP
# Скачиваем во временный файл, чтобы избежать повреждения при обрыве
temp_path = filepath + ".tmp"
with open(temp_path, "wb") as f:
for chunk in response.iter_content(chunk_size=1024 * 1024): # Чанки по 1 МБ
if chunk:
f.write(chunk)
# Переименовываем временный файл в целевой (атомарная операция)
os.rename(temp_path, filepath)
print(f"✓ Скачан {filename}")
return True
except (requests.RequestException, IOError) as e:
# Обработка ошибок сети или ввода-вывода
print(f"✗ Попытка {attempt}/{max_attempts} не удалась для {filename}: {e}")
# Удаляем повреждённые файлы
for path in [filepath + ".tmp", filepath]:
if os.path.exists(path):
try:
os.remove(path)
except OSError:
pass
# Экспоненциальная задержка перед следующей попыткой
if attempt < max_attempts:
time.sleep(2 ** attempt) # 2с, 4с, 8с, 16с
return False # Все попытки исчерпаны
def download_data(num_shards, download_workers=8):
"""Скачивает обучающие фрагменты + закреплённый валидационный фрагмент."""
# Создаём директорию для данных, если она не существует
os.makedirs(DATA_DIR, exist_ok=True)
# Определяем количество обучающих фрагментов (не больше максимума)
num_train = min(num_shards, MAX_SHARD)
ids = list(range(num_train)) # Индексы обучающих фрагментов: [0, 1, 2, ..., num_train-1]
# Добавляем валидационный фрагмент, если он ещё не в списке
if VAL_SHARD not in ids:
ids.append(VAL_SHARD)
# Подсчитываем уже скачанные файлы
existing = sum(1 for i in ids if os.path.exists(os.path.join(DATA_DIR, f"shard_{i:05d}.parquet")))
if existing == len(ids):
print(f"Данные: все {len(ids)} фрагментов уже скачаны в {DATA_DIR}")
return
needed = len(ids) - existing # Сколько ещё нужно скачать
print(f"Данные: скачиваем {needed} фрагментов ({existing} уже существуют)...")
# Ограничиваем число воркеров количеством необходимых файлов
workers = max(1, min(download_workers, needed))
# Параллельное скачивание через пул процессов
with Pool(processes=workers) as pool:
results = pool.map(download_single_shard, ids)
ok = sum(1 for r in results if r) # Считаем успешные загрузки
print(f"Данные: {ok}/{len(ids)} фрагментов готовы в {DATA_DIR}")
# ---------------------------------------------------------------------------
# Обучение токенизатора
# ---------------------------------------------------------------------------
def list_parquet_files():
"""Возвращает отсортированный список путей к Parquet-файлам в директории данных."""
# Фильтруем: только .parquet файлы, исключая временные .tmp
files = sorted(f for f in os.listdir(DATA_DIR) if f.endswith(".parquet") and not f.endswith(".tmp"))
return [os.path.join(DATA_DIR, f) for f in files] # Возвращаем полные пути
def text_iterator(max_chars=1_000_000_000, doc_cap=10_000):
"""
Генератор: выдаёт документы из обучающей выборки
(все фрагменты, кроме закреплённого валидационного).
max_chars=1_000_000_000 - лимит символов
doc_cap=10_000 - ограничитель длины документа для стабильности обучения
"""
# Исключаем валидационный файл из списка для обучения
parquet_paths = [p for p in list_parquet_files() if not p.endswith(VAL_FILENAME)]
nchars = 0 # Счётчик обработанных символов
for filepath in parquet_paths:
pf = pq.ParquetFile(filepath) # Открываем Parquet-файл
for rg_idx in range(pf.num_row_groups): # Итерируем по группам строк
rg = pf.read_row_group(rg_idx) # Читаем группу
for text in rg.column("text").to_pylist(): # Извлекаем тексты
# Ограничиваем длину документа (doc_cap) для стабильности обучения
doc = text[:doc_cap] if len(text) > doc_cap else text
nchars += len(doc)
yield doc # Возвращаем документ генератору
if nchars >= max_chars: # Прерываем при достижении лимита символов
return
def train_tokenizer():
"""Обучает BPE-токенизатор через rustbpe, сохраняет как pickle для tiktoken."""
tokenizer_pkl = os.path.join(TOKENIZER_DIR, "tokenizer.pkl") # Путь к основному файлу токенизатора
token_bytes_path = os.path.join(TOKENIZER_DIR, "token_bytes.pt") # Путь к таблице байтов на токен
# Если токенизатор уже обучен — пропускаем
if os.path.exists(tokenizer_pkl) and os.path.exists(token_bytes_path):
print(f"Токенизатор: уже обучен в {TOKENIZER_DIR}")
return
os.makedirs(TOKENIZER_DIR, exist_ok=True) # Создаём директорию токенизатора
# Проверяем наличие достаточного количества данных
parquet_files = list_parquet_files()
if len(parquet_files) < 2:
print("Токенизатор: нужно минимум 2 фрагмента данных (1 обуч.+1 валид.). Скачайте больше данных.")
sys.exit(1)
# --- Обучение через rustbpe ---
print("Токенизатор: обучаем BPE-токенизатор...")
t0 = time.time() # Засекаем время начала
tokenizer = rustbpe.Tokenizer() # Создаём экземпляр токенизатора
vocab_size_no_special = VOCAB_SIZE - len(SPECIAL_TOKENS) # Размер словаря без спец.токенов
# Запускаем обучение на потоке документов
tokenizer.train_from_iterator(text_iterator(), vocab_size_no_special, pattern=SPLIT_PATTERN)
# --- Сборка Encoding для tiktoken ---
pattern = tokenizer.get_pattern() # Получаем паттерн разбиения
# Преобразуем словари: ключи — байты, значения — ранги слияний
mergeable_ranks = {bytes(k): v for k, v in tokenizer.get_mergeable_ranks()}
tokens_offset = len(mergeable_ranks) # Смещение для спец.токенов
special_tokens = {name: tokens_offset + i for i, name in enumerate(SPECIAL_TOKENS)}
# Создаём объект Encoding в формате tiktoken
enc = tiktoken.Encoding(
name="rustbpe",
pat_str=pattern,
mergeable_ranks=mergeable_ranks,
special_tokens=special_tokens,
)
# Сохраняем токенизатор в pickle
with open(tokenizer_pkl, "wb") as f:
pickle.dump(enc, f)
t1 = time.time()
print(f"Токенизатор: обучен за {t1 - t0:.1f}с, сохранён в {tokenizer_pkl}")
# --- Построение lookup-таблицы token_bytes для оценки BPB ---
print("Токенизатор: строим таблицу token_bytes...")
special_set = set(SPECIAL_TOKENS) # Множество спец.токенов для быстрого поиска
token_bytes_list = []
for token_id in range(enc.n_vocab): # Для каждого токена в словаре
token_str = enc.decode([token_id]) # Декодируем токен в строку
if token_str in special_set:
token_bytes_list.append(0) # Спец.токены не учитываются в байтах
else:
# Считаем количество байт в UTF-8 представлении токена
token_bytes_list.append(len(token_str.encode("utf-8")))
# Сохраняем как PyTorch-тензор для быстрой загрузки на GPU
token_bytes_tensor = torch.tensor(token_bytes_list, dtype=torch.int32)
torch.save(token_bytes_tensor, token_bytes_path)
print(f"Токенизатор: таблица token_bytes сохранена в {token_bytes_path}")
# --- sanity check: проверка обратимости кодирования ---
test = "Hello world! Numbers: 123. Unicode: 你好"
encoded = enc.encode_ordinary(test)
decoded = enc.decode(encoded)
assert decoded == test, f"Ошибка кругового кодирования токенизатора: {test!r} -> {decoded!r}"
print(f"Токенизатор: проверка пройдена (размер словаря={enc.n_vocab})")
# ---------------------------------------------------------------------------
# Утилиты времени выполнения (для train.py)
# ---------------------------------------------------------------------------
class Tokenizer:
"""Минимальная обёртка токенизатора. Обучение обрабатывается выше."""
def __init__(self, enc):
self.enc = enc # Сохраняем объект tiktoken.Encoding
self.bos_token_id = enc.encode_single_token(BOS_TOKEN) # ID токена начала последовательности
@classmethod
def from_directory(cls, tokenizer_dir=TOKENIZER_DIR):
"""Загружает токенизатор из директории."""
with open(os.path.join(tokenizer_dir, "tokenizer.pkl"), "rb") as f:
enc = pickle.load(f) # Десериализуем Encoding
return cls(enc)
def get_vocab_size(self):
"""Возвращает размер словаря."""
return self.enc.n_vocab
def get_bos_token_id(self):
"""Возвращает ID токена BOS."""
return self.bos_token_id
def encode(self, text, prepend=None, num_threads=8):
"""
Кодирует текст в список ID токенов.
:param text: строка или список строк
:param prepend: токен для добавления в начало (строка или ID)
:param num_threads: число потоков для пакетного кодирования
"""
if prepend is not None:
# Преобразуем строку-токен в ID, если нужно
prepend_id = prepend if isinstance(prepend, int) else self.enc.encode_single_token(prepend)
if isinstance(text, str):
ids = self.enc.encode_ordinary(text) # Кодируем одну строку
if prepend is not None:
ids.insert(0, prepend_id) # Добавляем BOS в начало
return ids
elif isinstance(text, list):
# Пакетное кодирование с многопоточностью
ids = self.enc.encode_ordinary_batch(text, num_threads=num_threads)
if prepend is not None:
for row in ids:
row.insert(0, prepend_id)
return ids
else:
raise ValueError(f"Неверный тип входных данных: {type(text)}")
def decode(self, ids):
"""Декодирует список ID токенов обратно в строку."""
return self.enc.decode(ids)
def get_token_bytes(device="cpu"):
"""Загружает тензор с количеством байт на токен для оценки BPB."""
path = os.path.join(TOKENIZER_DIR, "token_bytes.pt")
with open(path, "rb") as f:
return torch.load(f, map_location=device) # Загрузка с автоматическим перемещением на устройство
def _document_batches(split, tokenizer_batch_size=128):
"""
Бесконечный итератор по пакетам документов из Parquet-файлов.
:param split: "train" или "val" — выборка данных
:param tokenizer_batch_size: размер пакета для токенизации
"""
parquet_paths = list_parquet_files()
assert len(parquet_paths) > 0, "Parquet-файлы не найдены. Сначала запустите prepare.py."
val_path = os.path.join(DATA_DIR, VAL_FILENAME)
# Выбираем файлы в зависимости от выборки
if split == "train":
parquet_paths = [p for p in parquet_paths if p != val_path] # Исключаем валидацию
assert len(parquet_paths) > 0, "Обучающие фрагменты не найдены."
else:
parquet_paths = [val_path] # Только валидационный файл
epoch = 1 # Счётчик эпох
while True: # Бесконечный цикл — данные подаются потоком
for filepath in parquet_paths:
pf = pq.ParquetFile(filepath)
for rg_idx in range(pf.num_row_groups):
rg = pf.read_row_group(rg_idx)
batch = rg.column('text').to_pylist() # Извлекаем тексты
# Делим на мини-пакеты для токенизации
for i in range(0, len(batch), tokenizer_batch_size):
yield batch[i:i+tokenizer_batch_size], epoch
epoch += 1 # Увеличиваем номер эпохи после прохода по всем файлам
def make_dataloader(tokenizer, B, T, split, buffer_size=1000):
"""
Загрузчик данных с выравниванием по BOS и best-fit упаковкой.
• Каждая строка начинается с токена BOS.
• Документы упаковываются методом best-fit для минимизации обрезки.
• Если ни один документ не помещается в остаток — обрезается самый короткий.
• 100% утилизация (без паддинга).
:param tokenizer: объект Tokenizer
:param B: размер батча (batch size)
:param T: длина последовательности (sequence length)
:param split: "train" или "val"
:param buffer_size: размер буфера документов для упаковки
"""
assert split in ["train", "val"]
row_capacity = T + 1 # +1 для токена BOS
batches = _document_batches(split) # Источник пакетов документов
bos_token = tokenizer.get_bos_token_id()
doc_buffer = [] # Буфер токенизированных документов
epoch = 1
def refill_buffer():
"""Пополняет буфер документов, токенизируя новый пакет."""
nonlocal epoch
doc_batch, epoch = next(batches) # Получаем следующий пакет
token_lists = tokenizer.encode(doc_batch, prepend=bos_token) # Токенизация с BOS
doc_buffer.extend(token_lists) # Добавляем в буфер
# Предварительное выделение буферов: [inputs (B*T) | targets (B*T)]
row_buffer = torch.empty((B, row_capacity), dtype=torch.long) # Буфер для одной строки батча
cpu_buffer = torch.empty(2 * B * T, dtype=torch.long, pin_memory=True) # CPU-буфер с pinned memory
gpu_buffer = torch.empty(2 * B * T, dtype=torch.long, device="cuda") # GPU-буфер
cpu_inputs = cpu_buffer[:B * T].view(B, T) # Вид на входные данные
cpu_targets = cpu_buffer[B * T:].view(B, T) # Вид на целевые данные
inputs = gpu_buffer[:B * T].view(B, T) # GPU-вид входов
targets = gpu_buffer[B * T:].view(B, T) # GPU-вид целей
while True: # Бесконечный генератор батчей
for row_idx in range(B): # Для каждой строки в батче
pos = 0 # Позиция в строке
while pos < row_capacity:
# Пополняем буфер, если документов мало
while len(doc_buffer) < buffer_size:
refill_buffer()
remaining = row_capacity - pos # Сколько ещё места осталось
# --- Best-Fit: ищем наибольший документ, который помещается целиком ---
best_idx = -1
best_len = 0
for i, doc in enumerate(doc_buffer):
doc_len = len(doc)
if doc_len <= remaining and doc_len > best_len:
best_idx = i
best_len = doc_len
if best_idx >= 0:
# Документ помещается — добавляем его целиком
doc = doc_buffer.pop(best_idx)
row_buffer[row_idx, pos:pos + len(doc)] = torch.tensor(doc, dtype=torch.long)
pos += len(doc)
else:
# Ни один документ не помещается — обрезаем самый короткий
shortest_idx = min(range(len(doc_buffer)), key=lambda i: len(doc_buffer[i]))
doc = doc_buffer.pop(shortest_idx)
row_buffer[row_idx, pos:pos + remaining] = torch.tensor(doc[:remaining], dtype=torch.long)
pos += remaining # Заполняем остаток обрезанным документом
# Копируем данные: CPU -> GPU (асинхронно)
cpu_inputs.copy_(row_buffer[:, :-1]) # Входы: все токены, кроме последнего
cpu_targets.copy_(row_buffer[:, 1:]) # Цели: все токены, кроме первого (сдвиг на 1)
gpu_buffer.copy_(cpu_buffer, non_blocking=True) # Асинхронная копия на GPU
yield inputs, targets, epoch # Возвращаем батч и номер эпохи
# ---------------------------------------------------------------------------
# Оценка качества (фиксированная метрика — НЕ ИЗМЕНЯТЬ)
# ---------------------------------------------------------------------------
@torch.no_grad() # Отключаем вычисление градиентов для экономии памяти
def evaluate_bpb(model, tokenizer, batch_size):
"""
Bits Per Byte (BPB): метрика оценки, независимая от размера словаря.
Алгоритм:
1. Суммируем пер-токен кросс-энтропию (в натах)
2. Суммируем длины целевых токенов в байтах
3. Исключаем спец.токены (длина байт = 0) из обоих сумм
4. Конвертируем наты/байт → биты/байт через деление на ln(2)
Использует фиксированный MAX_SEQ_LEN для сравнимости результатов.
"""
token_bytes = get_token_bytes(device="cuda") # Загружаем таблицу байт/токен на GPU
val_loader = make_dataloader(tokenizer, batch_size, MAX_SEQ_LEN, "val") # Валидационный загрузчик
steps = EVAL_TOKENS // (batch_size * MAX_SEQ_LEN) # Сколько шагов нужно для покрытия EVAL_TOKENS
total_nats = 0.0 # Накопленная сумма нат
total_bytes = 0 # Накопленное количество байт
for _ in range(steps):
x, y, _ = next(val_loader) # Получаем батч: входы, цели, эпоха
# Вычисляем потери для каждого токена отдельно
loss_flat = model(x, y, reduction='none').view(-1) # [B*T]
y_flat = y.view(-1) # [B*T] — целевые токены
nbytes = token_bytes[y_flat] # Количество байт для каждого целевого токена
mask = nbytes > 0 # Маска: исключаем спец.токены (у них 0 байт)
# Суммируем только учтённые потери и байты
total_nats += (loss_flat * mask).sum().item()
total_bytes += nbytes.sum().item()
# Конвертация: наты → биты (деление на ln(2)), затем биты/байт
return total_nats / (math.log(2) * total_bytes)
# ---------------------------------------------------------------------------
# Точка входа (main)
# ---------------------------------------------------------------------------
if __name__ == "__main__":
# Парсинг аргументов командной строки
parser = argparse.ArgumentParser(description="Подготовка данных и токенизатора для autoresearch")
parser.add_argument("--num-shards", type=int, default=10,
help="Число обучающих фрагментов для скачивания (-1 = все). Валидационный фрагмент всегда закрепляется.")
parser.add_argument("--download-workers", type=int, default=8,
help="Число параллельных воркеров для скачивания")
args = parser.parse_args()
# Обработка флага "все фрагменты"
num_shards = MAX_SHARD if args.num_shards == -1 else args.num_shards
print(f"Директория кэша: {CACHE_DIR}")
print()
# Шаг 1: Скачивание данных
download_data(num_shards, download_workers=args.download_workers)
print()
# Шаг 2: Обучение токенизатора
train_tokenizer()
print()
print("✓ Готово! Можно запускать обучение.")Пояснения по константам:
Базовые константы:
Константа | Значение | Пояснение |
|---|---|---|
| 2048 | Максимальная длина входной последовательности для модели |
| 300 | Ограничение времени на один эксперимент (для авто-исследования) |
| 20 971 520 | Объём данных для расчёта метрики качества (BPB) |
Константы конфигурации:
Константа | Значение | Пояснение |
|---|---|---|
|
| Корневая директория для кэширования |
|
| Папка для хранения скачанных Parquet-файлов |
|
| Папка для сохранения обученного токенизатора |
| URL HuggingFace | Источник данных для скачивания |
| 6542 | Общее количество доступных фрагментов данных |
| 6542 | Индекс фрагмента, закреплённого для валидации (не используется в обучении) |
| 8192 | Целевой размер словаря BPE-токенизатора |
Константы паттерна разбиения BPE:
Константа | Пояснение |
|---|---|
| Регулярное выражение для пред-токенизации (определяет границы слов/субслов) |
| Список из 4 зарезервированных токенов для служебных целей |
| Токен, добавляемый в начало каждой последовательности |
Еще раз повторим ключевые термины:
Термин | Значение |
|---|---|
Shard | Фрагмент большого датасета, разбитого на части для параллельной обработки |
BPE (Byte Pair Encoding) | Алгоритм субсловной токенизации, используемый в GPT-моделях |
BPB (Bits Per Byte) | Метрика качества: сколько бит информации требуется для кодирования одного байта текста. Меньше = лучше |
BOS (Begin Of Sequence) | Специальный токен, маркирующий начало последовательности |
Best-Fit packing | Алгоритм упаковки: выбирает наибольший документ, который помещается в остаток места |
Nats | Единица измерения энтропии на основе натурального логарифма (1 nat = 1/ln(2) бит) |
Pinned memory | Память CPU, закреплённая для ускоренной передачи на GPU через PCIe |
Обратите внимание на особенности реализации:
Устойчивость к сбоям: скачивание с экспоненциальной задержкой и атомарным переименованием файлов
Эффективная упаковка: best-fit алгоритм минимизирует обрезку документов и обеспечивает 100% утилизацию контекста
Асинхронная загрузка: предвыборка данных на CPU + pinned memory + non-blocking copy на GPU
Воспроизводимость: закреплённый валидационный shard гарантирует сравнимость метрик между экспериментами
Масштабируемость: параллельное скачивание и потоковая обработка данных позволяют работать с датасетами в сотни ГБ
Анализ кода файла train.py
Назначение файла: скрипт предобучения модели для autoresearch. Однофайловая реализация для одного GPU, упрощённая версия из nanochat. Этот файл многократно модифицируется агентом, для получения максимального BPE.
train.py — это высокооптимизированный, исследовательский скрипт для быстрого прототипирования архитектурных изменений. Он сочетает передовые техники (Muon, Flash Attention 3, RoPE, ResFormer) с практическими оптимизациями (torch.compile, fused kernels, memory management) для максимального использования ресурсов одного GPU.
"""
Скрипт предобучения Autoresearch. Однофайловая реализация для одного GPU.
Выбранные и упрощённые компоненты из nanochat.
Использование: uv run train.py
"""
import os
# Включаем расширяемые сегменты для аллокатора памяти PyTorch (улучшает фрагментацию)
os.environ["PYTORCH_ALLOC_CONF"] = "expandable_segments:True"
# Отключаем прогресс-бары HuggingFace для чистого вывода в логах
os.environ["HF_HUB_DISABLE_PROGRESS_BARS"] = "1"
import gc # Управление сборщиком мусора (для минимизации пауз)
import math # Математические функции
import time # Замер времени выполнения
from dataclasses import dataclass, asdict # Декораторы для конфигураций
import torch # Фреймворк глубокого обучения
import torch.nn as nn # Модули нейросетей
import torch.nn.functional as F # Функциональные операции
from kernels import get_kernel # Импорт кастомных CUDA-ядер
cap = torch.cuda.get_device_capability() # Получаем возможности GPU
# Выбираем репозиторий flash-attention-3 в зависимости от архитектуры GPU:
# Hopper (9.0) → varunneal, остальные → kernels-community
repo = "varunneal/flash-attention-3" if cap == (9, 0) else "kernels-community/flash-attn3"
fa3 = get_kernel(repo).flash_attn_interface # Импорт функции flash-attention
# Импорт констант и утилит из prepare.py
from prepare import MAX_SEQ_LEN, TIME_BUDGET, Tokenizer, make_dataloader, evaluate_bpb
# ---------------------------------------------------------------------------
# GPT Model
# ---------------------------------------------------------------------------
@dataclass
class GPTConfig:
"""Конфигурация архитектуры трансформера (использует dataclass для удобства)."""
sequence_len: int = 2048 # Длина контекстной последовательности
vocab_size: int = 32768 # Размер словаря токенов
n_layer: int = 12 # Количество слоёв трансформера
n_head: int = 6 # Количество голов внимания
n_kv_head: int = 6 # Количество KV-голов (для GQA; =n_head означает MHA)
n_embd: int = 768 # Размерность эмбеддингов (d_model)
window_pattern: str = "SSSL" # Паттерн окон внимания: S=short, L=long
# RMSNorm: Упрощённая нормализация, которая делит активации на их RMS
# (среднеквадратичное значение), без вычисления среднего и без обучаемого смещения.
# Экономит вычисления и память.
def norm(x):
"""Применяет RMSNorm (Root Mean Square Layer Normalization) без смещения."""
return F.rms_norm(x, (x.size(-1),))
def has_ve(layer_idx, n_layer):
"""
Возвращает True, если слой должен использовать Value Embedding (VE).
Схема: чередующиеся слои, последний слой всегда включён.
layer_idx: Индекс текущего слоя (0...n_layer-1)
n_layer: Общее количество слоёв
"""
return layer_idx % 2 == (n_layer - 1) % 2
# RoPE (Rotary Position Embedding):
# Кодирует позицию токена через вращение векторов в комплексной плоскости.
# Позволяет модели обобщаться на последовательности длиннее обучающих.
def apply_rotary_emb(x, cos, sin):
"""
Применяет вращательные позиционные эмбеддинги (RoPE) к тензору внимания.
Разделяет head_dim пополам, применяет вращение через cos/sin, конкатенирует обратно.
"""
assert x.ndim == 4 # [B, T, n_head, head_dim]
d = x.shape[3] // 2 # Половина размерности головы
x1, x2 = x[..., :d], x[..., d:] # Разделяем на две части
y1 = x1 * cos + x2 * sin # Первая половина вращения
y2 = x1 * (-sin) + x2 * cos # Вторая половина
return torch.cat([y1, y2], 3) # Собираем обратно
class CausalSelfAttention(nn.Module):
"""Слой внимания с:
• GQA (Grouped Query Attention) через n_kv_head <= n_head
• Value Embedding (ResFormer) — добавление обучаемых value-векторов
• Flash Attention 3 для ускорения
• RMSNorm перед проекциями
• Скользящее окно внимания (sliding window)
"""
def __init__(self, config, layer_idx):
super().__init__()
self.n_head = config.n_head
self.n_kv_head = config.n_kv_head
self.n_embd = config.n_embd
self.head_dim = self.n_embd // self.n_head
# Проверки целостности конфигурации
assert self.n_embd % self.n_head == 0
assert self.n_kv_head <= self.n_head and self.n_head % self.n_kv_head == 0
# Линейные проекции для Q, K, V (без смещения для стабильности)
self.c_q = nn.Linear(self.n_embd, self.n_head * self.head_dim, bias=False)
self.c_k = nn.Linear(self.n_embd, self.n_kv_head * self.head_dim, bias=False)
self.c_v = nn.Linear(self.n_embd, self.n_kv_head * self.head_dim, bias=False)
self.c_proj = nn.Linear(self.n_embd, self.n_embd, bias=False) # Выходная проекция
# Value Embedding gate (ResFormer): только для слоёв с VE
self.ve_gate_channels = 32 # Количество каналов для вычисления гейта
self.ve_gate = nn.Linear(self.ve_gate_channels, self.n_kv_head, bias=False) if has_ve(layer_idx, config.n_layer) else None
def forward(self, x, ve, cos_sin, window_size):
"""
Прямой проход слоя внимания.
:param x: входные активации [B, T, n_embd]
:param ve: value embedding для текущего слоя (или None)
:param cos_sin: кортеж (cos, sin) для RoPE
:param window_size: кортеж (window, 0) для sliding window attention
"""
B, T, C = x.size()
# Проекция в Q, K, V и рескейп к форме с головами
q = self.c_q(x).view(B, T, self.n_head, self.head_dim)
k = self.c_k(x).view(B, T, self.n_kv_head, self.head_dim)
v = self.c_v(x).view(B, T, self.n_kv_head, self.head_dim)
# === Value Residual (ResFormer) ===
# Смешиваем обучаемые value-эмбеддинги с входными v через гейт, зависящий от x
if ve is not None:
ve = ve.view(B, T, self.n_kv_head, self.head_dim)
# Гейт: 2 * sigmoid(...) даёт диапазон [0, 2], где 1.0 = нейтрально
gate = 2 * torch.sigmoid(self.ve_gate(x[..., :self.ve_gate_channels]))
v = v + gate.unsqueeze(-1) * ve # Broadcasting по head_dim
# Применяем RoPE к запросам и ключам
cos, sin = cos_sin
q, k = apply_rotary_emb(q, cos, sin), apply_rotary_emb(k, cos, sin)
# RMSNorm перед attention (стабилизирует вычисления)
q, k = norm(q), norm(k)
# Flash Attention 3: эффективное вычисление causal attention с окном
y = fa3.flash_attn_func(q, k, v, causal=True, window_size=window_size)
# Рескейп и выходная проекция
y = y.contiguous().view(B, T, -1)
y = self.c_proj(y)
return y
class MLP(nn.Module):
"""
MLP-блок с расширением 4× и активацией: F.relu(x).square() (т.н. 'square-ReLU').
Square-ReLU: Комбинация ReLU(x)² даёт более резкую нелинейность, чем обычный ReLU, что может улучшать обучение в некоторых архитектурах.
"""
def __init__(self, config):
super().__init__()
self.c_fc = nn.Linear(config.n_embd, 4 * config.n_embd, bias=False) # Расширение
self.c_proj = nn.Linear(4 * config.n_embd, config.n_embd, bias=False) # Проекция обратно
def forward(self, x):
x = self.c_fc(x)
x = F.relu(x).square() # Квадратный ReLU: усиливает большие активации
x = self.c_proj(x)
return x
class Block(nn.Module):
"""Полный блок трансформера: Attention + MLP с остаточными связями и пред-нормализацией."""
def __init__(self, config, layer_idx):
super().__init__()
self.attn = CausalSelfAttention(config, layer_idx)
self.mlp = MLP(config)
def forward(self, x, ve, cos_sin, window_size):
# Остаточная связь + пред-нормализация (Pre-LN)
x = x + self.attn(norm(x), ve, cos_sin, window_size)
x = x + self.mlp(norm(x))
return x
class GPT(nn.Module):
"""Полная GPT-модель с:
• Чередующимися Value Embeddings (ResFormer)
• Скользящими окнами внимания
• Per-layer scalars (resid_lambdas, x0_lambdas) для динамического масштабирования остатков
• Инициализацией весов по схеме nanochat
"""
def __init__(self, config):
super().__init__()
self.config = config
self.window_sizes = self._compute_window_sizes(config) # Паттерн окон по слоям
self.transformer = nn.ModuleDict({
"wte": nn.Embedding(config.vocab_size, config.n_embd), # Токен-эмбеддинги
"h": nn.ModuleList([Block(config, i) for i in range(config.n_layer)]), # Слои
})
self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False) # Выход на словарь
# Per-layer learnable scalars для остаточных связей
self.resid_lambdas = nn.Parameter(torch.ones(config.n_layer)) # Масштаб текущего x
self.x0_lambdas = nn.Parameter(torch.zeros(config.n_layer)) # Масштаб начального x0
# Value embeddings: только для слоёв, где has_ve=True
head_dim = config.n_embd // config.n_head
kv_dim = config.n_kv_head * head_dim
self.value_embeds = nn.ModuleDict({
str(i): nn.Embedding(config.vocab_size, kv_dim)
for i in range(config.n_layer) if has_ve(i, config.n_layer)
})
# Rotary embeddings: предвычисляем cos/sin для длинной последовательности
self.rotary_seq_len = config.sequence_len * 10 # Запас для обобщения на длинные контексты
cos, sin = self._precompute_rotary_embeddings(self.rotary_seq_len, head_dim)
self.register_buffer("cos", cos, persistent=False) # Не сохраняем в state_dict
self.register_buffer("sin", sin, persistent=False)
@torch.no_grad()
def init_weights(self):
"""Инициализация весов по схеме nanochat (устойчивая для глубоких сетей)."""
# Эмбеддинги и выходной слой
torch.nn.init.normal_(self.transformer.wte.weight, mean=0.0, std=1.0)
torch.nn.init.normal_(self.lm_head.weight, mean=0.0, std=0.001)
# Transformer blocks
n_embd = self.config.n_embd
s = 3**0.5 * n_embd**-0.5
for block in self.transformer.h:
torch.nn.init.uniform_(block.attn.c_q.weight, -s, s)
torch.nn.init.uniform_(block.attn.c_k.weight, -s, s)
torch.nn.init.uniform_(block.attn.c_v.weight, -s, s)
torch.nn.init.zeros_(block.attn.c_proj.weight) # Нулевая инициализация выхода внимания
torch.nn.init.uniform_(block.mlp.c_fc.weight, -s, s)
torch.nn.init.zeros_(block.mlp.c_proj.weight) # Нулевая инициализация выхода MLP
# Per-layer scalars
self.resid_lambdas.fill_(1.0) # Начинаем с полного использования текущего x
self.x0_lambdas.fill_(0.1) # Небольшой вклад начального x0
# Value embeddings
for ve in self.value_embeds.values():
torch.nn.init.uniform_(ve.weight, -s, s)
# Gate weights → 0 (sigmoid(0)=0.5, ×2 = 1.0 → нейтральное смешивание)
for block in self.transformer.h:
if block.attn.ve_gate is not None:
torch.nn.init.zeros_(block.attn.ve_gate.weight)
# Обновляем буферы RoPE
head_dim = self.config.n_embd // self.config.n_head
cos, sin = self._precompute_rotary_embeddings(self.rotary_seq_len, head_dim)
self.cos, self.sin = cos, sin
# Конвертируем эмбеддинги в bfloat16 для экономии памяти
self.transformer.wte.to(dtype=torch.bfloat16)
for ve in self.value_embeds.values():
ve.to(dtype=torch.bfloat16)
def _precompute_rotary_embeddings(self, seq_len, head_dim, base=10000, device=None):
"""Предвычисляет cos/sin для вращательных эмбеддингов (RoPE)."""
if device is None:
device = self.transformer.wte.weight.device
# Частоты для каждой пары каналов: 1 / base^(2i/d)
channel_range = torch.arange(0, head_dim, 2, dtype=torch.float32, device=device)
inv_freq = 1.0 / (base ** (channel_range / head_dim))
# Временная ось
t = torch.arange(seq_len, dtype=torch.float32, device=device)
freqs = torch.outer(t, inv_freq)
cos, sin = freqs.cos(), freqs.sin()
cos, sin = cos.bfloat16(), sin.bfloat16() # Конвертация для экономии памяти
cos, sin = cos[None, :, None, :], sin[None, :, None, :]
return cos, sin
def _compute_window_sizes(self, config):
"""Вычисляет размер окна внимания для каждого слоя по паттерну (S=short, L=long)."""
pattern = config.window_pattern.upper()
assert all(c in "SL" for c in pattern)
long_window = config.sequence_len # Полное внимание
short_window = long_window // 2 # Половинное окно
char_to_window = {"L": (long_window, 0), "S": (short_window, 0)}
window_sizes = []
for layer_idx in range(config.n_layer):
char = pattern[layer_idx % len(pattern)]
window_sizes.append(char_to_window[char])
# Последний слой всегда получает полное окно (для глобального контекста)
window_sizes[-1] = (long_window, 0)
return window_sizes
def estimate_flops(self):
"""Оценивает FLOPs на токен (forward + backward) для расчёта MFU."""
nparams = sum(p.numel() for p in self.parameters())
# Исключаем из подсчёта параметры, не участвующие в матричных умножениях
value_embeds_numel = sum(ve.weight.numel() for ve in self.value_embeds.values())
nparams_exclude = (self.transformer.wte.weight.numel() + value_embeds_numel +
self.resid_lambdas.numel() + self.x0_lambdas.numel())
h = self.config.n_head
q = self.config.n_embd // self.config.n_head
t = self.config.sequence_len
# Подсчёт FLOPs для attention с учётом sliding window
attn_flops = 0
for window_size in self.window_sizes:
window = window_size[0]
effective_seq = t if window < 0 else min(window, t)
attn_flops += 12 * h * q * effective_seq
# 6× для forward+backward, плюс attention-компонент
return 6 * (nparams - nparams_exclude) + attn_flops
def num_scaling_params(self):
"""Возвращает словарь с количеством параметров по группам (для отчётов)."""
wte = sum(p.numel() for p in self.transformer.wte.parameters())
value_embeds = sum(p.numel() for p in self.value_embeds.parameters())
lm_head = sum(p.numel() for p in self.lm_head.parameters())
transformer_matrices = sum(p.numel() for p in self.transformer.h.parameters())
scalars = self.resid_lambdas.numel() + self.x0_lambdas.numel()
total = wte + value_embeds + lm_head + transformer_matrices + scalars
return {
'wte': wte, 'value_embeds': value_embeds, 'lm_head': lm_head,
'transformer_matrices': transformer_matrices, 'scalars': scalars, 'total': total,
}
def setup_optimizer(self, unembedding_lr=0.004, embedding_lr=0.2, matrix_lr=0.02,
weight_decay=0.0, adam_betas=(0.8, 0.95), scalar_lr=0.5):
"""
Настраивает гибридный оптимизатор:
• AdamW для эмбеддингов, lm_head и скаляров
• Muon для матричных весов трансформера
• Масштабирование LR ∝ 1/√d_model
"""
model_dim = self.config.n_embd
# Группируем параметры по типу
matrix_params = list(self.transformer.h.parameters()) # Матрицы внимания и MLP
value_embeds_params = list(self.value_embeds.parameters())
embedding_params = list(self.transformer.wte.parameters())
lm_head_params = list(self.lm_head.parameters())
resid_params = [self.resid_lambdas]
x0_params = [self.x0_lambdas]
# Проверка: все параметры распределены
assert len(list(self.parameters())) == (len(matrix_params) + len(embedding_params) +
len(lm_head_params) + len(value_embeds_params) + len(resid_params) + len(x0_params))
# Масштабируем LR в зависимости от размерности модели (калибровка на d_model=768)
dmodel_lr_scale = (model_dim / 768) ** -0.5
print(f"Scaling AdamW LRs by 1/sqrt({model_dim}/768) = {dmodel_lr_scale:.6f}")
# Формируем группы параметров для оптимизатора
param_groups = [
dict(kind='adamw', params=lm_head_params, lr=unembedding_lr * dmodel_lr_scale, betas=adam_betas, eps=1e-10, weight_decay=0.0),
dict(kind='adamw', params=embedding_params, lr=embedding_lr * dmodel_lr_scale, betas=adam_betas, eps=1e-10, weight_decay=0.0),
dict(kind='adamw', params=value_embeds_params, lr=embedding_lr * dmodel_lr_scale, betas=adam_betas, eps=1e-10, weight_decay=0.0),
dict(kind='adamw', params=resid_params, lr=scalar_lr * 0.01, betas=adam_betas, eps=1e-10, weight_decay=0.0),
dict(kind='adamw', params=x0_params, lr=scalar_lr, betas=(0.96, 0.95), eps=1e-10, weight_decay=0.0),
]
# Группируем матричные параметры по форме для Muon (одинаковые формы → один оптимизатор)
for shape in sorted({p.shape for p in matrix_params}):
group_params = [p for p in matrix_params if p.shape == shape]
param_groups.append(dict(
kind='muon', params=group_params, lr=matrix_lr,
momentum=0.95, ns_steps=5, beta2=0.95, weight_decay=weight_decay,
))
optimizer = MuonAdamW(param_groups)
# Сохраняем начальные LR для расчёта расписания
for group in optimizer.param_groups:
group["initial_lr"] = group["lr"]
return optimizer
def forward(self, idx, targets=None, reduction='mean'):
"""
Прямой проход модели.
:param idx: входные токены [B, T]
:param targets: целевые токены для расчёта loss (опционально)
:param reduction: 'mean' или 'none' для функции потерь
"""
B, T = idx.size()
assert T <= self.cos.size(1) # Проверка: не превышаем предвычисленные RoPE
cos_sin = self.cos[:, :T], self.sin[:, :T] # Обрезаем под текущую длину
# Токен-эмбеддинги + нормализация
x = self.transformer.wte(idx)
x = norm(x)
x0 = x # Сохраняем начальное состояние для x0_lambdas
# Проход по слоям трансформера
for i, block in enumerate(self.transformer.h):
# Динамическое масштабирование остаточных связей
x = self.resid_lambdas[i] * x + self.x0_lambdas[i] * x0
# Получаем value embedding для текущего слоя (если есть)
ve = self.value_embeds[str(i)](idx) if str(i) in self.value_embeds else None
# Прямой проход блока
x = block(x, ve, cos_sin, self.window_sizes[i])
# Финальная нормализация
x = norm(x)
# Выходной слой с tanh-softcapping (стабилизация логитов)
softcap = 15
logits = self.lm_head(x)
logits = logits.float() # Вычисления в float32 для точности
logits = softcap * torch.tanh(logits / softcap) # Ограничение диапазона
# Расчёт потерь, если заданы targets
if targets is not None:
loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1),
ignore_index=-1, reduction=reduction)
return loss
return logits
# ---------------------------------------------------------------------------
# Оптимизатор (MuonAdamW, только для единичного GPU)
# ---------------------------------------------------------------------------
# Коэффициенты полиномиальной аппроксимации для итеративной ортогонализации матриц
# Используется в Muon для стабилизации обновления весов
polar_express_coeffs = [
(8.156554524902461, -22.48329292557795, 15.878769915207462),
(4.042929935166739, -2.808917465908714, 0.5000178451051316),
(3.8916678022926607, -2.772484153217685, 0.5060648178503393),
(3.285753657755655, -2.3681294933425376, 0.46449024233003106),
(2.3465413258596377, -1.7097828382687081, 0.42323551169305323),
]
@torch.compile(dynamic=False, fullgraph=True)
def adamw_step_fused(p, grad, exp_avg, exp_avg_sq, step_t, lr_t, beta1_t, beta2_t, eps_t, wd_t):
"""
Скомпилированная (torch.compile) реализация шага AdamW.
Все операции в одном графе для минимизации накладных расходов.
"""
# Weight decay (применяется до обновления момента)
p.mul_(1 - lr_t * wd_t)
# Обновление моментов
exp_avg.lerp_(grad, 1 - beta1_t) # exp_avg = beta1*exp_avg + (1-beta1)*grad
exp_avg_sq.lerp_(grad.square(), 1 - beta2_t)
# Коррекция смещения (bias correction)
bias1 = 1 - beta1_t ** step_t
bias2 = 1 - beta2_t ** step_t
# Вычисление шага
denom = (exp_avg_sq / bias2).sqrt() + eps_t
step_size = lr_t / bias1
# Обновление параметра
p.add_(exp_avg / denom, alpha=-step_size)
@torch.compile(dynamic=False, fullgraph=True)
def muon_step_fused(stacked_grads, stacked_params, momentum_buffer, second_momentum_buffer,
momentum_t, lr_t, wd_t, beta2_t, ns_steps, red_dim):
"""
Скомпилированная реализация шага Muon-оптимизатора.
Особенности:
• Nesterov momentum
• Polar Express ортогонализация (итеративная аппроксимация SVD)
• NorMuon variance reduction (нормализация по второму моменту)
• Cautious weight decay (только для согласованных знаков градиента и параметра)
"""
# === 1. Nesterov momentum ===
momentum = momentum_t.to(stacked_grads.dtype)
momentum_buffer.lerp_(stacked_grads, 1 - momentum)
g = stacked_grads.lerp_(momentum_buffer, momentum) # g = grad + momentum*(momentum_buffer - grad)
# === 2. Polar Express ортогонализация ===
X = g.bfloat16()
# Нормализация по Фробениусу
X = X / (X.norm(dim=(-2, -1), keepdim=True) * 1.02 + 1e-6)
# Итеративная ортогонализация через полиномиальную аппроксимацию
if g.size(-2) > g.size(-1): # "Высокая" матрица
for a, b, c in polar_express_coeffs[:ns_steps]:
A = X.mT @ X # X^T X
B = b * A + c * (A @ A)
X = a * X + X @ B
else: # "Широкая" матрица
for a, b, c in polar_express_coeffs[:ns_steps]:
A = X @ X.mT # X X^T
B = b * A + c * (A @ A)
X = a * X + B @ X
g = X # Ортогонализованный градиент
# === 3. NorMuon: нормализация по второму моменту ===
beta2 = beta2_t.to(g.dtype)
v_mean = g.float().square().mean(dim=red_dim, keepdim=True) # Средний квадрат по строкам/столбцам
red_dim_size = g.size(red_dim)
v_norm_sq = v_mean.sum(dim=(-2, -1), keepdim=True) * red_dim_size
v_norm = v_norm_sq.sqrt()
second_momentum_buffer.lerp_(v_mean.to(dtype=second_momentum_buffer.dtype), 1 - beta2)
step_size = second_momentum_buffer.clamp_min(1e-10).rsqrt()
scaled_sq_sum = (v_mean * red_dim_size) * step_size.float().square()
v_norm_new = scaled_sq_sum.sum(dim=(-2, -1), keepdim=True).sqrt()
final_scale = step_size * (v_norm / v_norm_new.clamp_min(1e-10))
g = g * final_scale.to(g.dtype) # Масштабируем градиент
# === 4. Cautious weight decay + обновление ===
lr = lr_t.to(g.dtype)
wd = wd_t.to(g.dtype)
# Применяем weight decay только если знак градиента и параметра совпадает
mask = (g * stacked_params) >= 0
stacked_params.sub_(lr * g + lr * wd * stacked_params * mask)
class MuonAdamW(torch.optim.Optimizer):
"""
Гибридный оптимизатор:
• Muon для 2D матричных параметров (веса внимания и MLP)
• AdamW для эмбеддингов, скаляров и lm_head
Особенности:
• Все шаги скомпилированы через torch.compile
• Использует 0-D CPU-тензоры для гиперпараметров (избегает рекомпиляции при изменении LR)
"""
def __init__(self, param_groups):
super().__init__(param_groups, defaults={})
# 0-D CPU-тензоры для гиперпараметров (избегают рекомпиляции torch.compile)
self._adamw_step_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._adamw_lr_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._adamw_beta1_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._adamw_beta2_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._adamw_eps_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._adamw_wd_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._muon_momentum_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._muon_lr_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._muon_wd_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
self._muon_beta2_t = torch.tensor(0.0, dtype=torch.float32, device="cpu")
def _step_adamw(self, group):
"""Выполняет шаг AdamW для одной группы параметров."""
for p in group['params']:
if p.grad is None:
continue
grad = p.grad
state = self.state[p]
# Инициализация состояния при первом шаге
if not state:
state['step'] = 0
state['exp_avg'] = torch.zeros_like(p)
state['exp_avg_sq'] = torch.zeros_like(p)
state['step'] += 1
# Обновляем 0-D тензоры гиперпараметров
self._adamw_step_t.fill_(state['step'])
self._adamw_lr_t.fill_(group['lr'])
self._adamw_beta1_t.fill_(group['betas'][0])
self._adamw_beta2_t.fill_(group['betas'][1])
self._adamw_eps_t.fill_(group['eps'])
self._adamw_wd_t.fill_(group['weight_decay'])
# Вызываем скомпилированную функцию шага
adamw_step_fused(p, grad, state['exp_avg'], state['exp_avg_sq'],
self._adamw_step_t, self._adamw_lr_t, self._adamw_beta1_t,
self._adamw_beta2_t, self._adamw_eps_t, self._adamw_wd_t)
def _step_muon(self, group):
"""Выполняет шаг Muon для группы матричных параметров."""
params = group['params']
if not params:
return
p = params[0]
state = self.state[p]
num_params = len(params)
shape, device, dtype = p.shape, p.device, p.dtype
# Инициализация буферов при первом шаге
if "momentum_buffer" not in state:
state["momentum_buffer"] = torch.zeros(num_params, *shape, dtype=dtype, device=device)
if "second_momentum_buffer" not in state:
# Форма второго момента зависит от ориентации матрицы
state_shape = (num_params, shape[-2], 1) if shape[-2] >= shape[-1] else (num_params, 1, shape[-1])
state["second_momentum_buffer"] = torch.zeros(state_shape, dtype=dtype, device=device)
red_dim = -1 if shape[-2] >= shape[-1] else -2 # Ось для редукции
# Стек градиентов и параметров для векторизованного обновления
stacked_grads = torch.stack([p.grad for p in params])
stacked_params = torch.stack(params)
# Обновляем гиперпараметры
self._muon_momentum_t.fill_(group["momentum"])
self._muon_beta2_t.fill_(group["beta2"] if group["beta2"] is not None else 0.0)
# Масштабирование LR по форме матрицы (для "широких" матриц)
self._muon_lr_t.fill_(group["lr"] * max(1.0, shape[-2] / shape[-1])**0.5)
self._muon_wd_t.fill_(group["weight_decay"])
# Вызываем скомпилированную функцию шага Muon
muon_step_fused(stacked_grads, stacked_params,
state["momentum_buffer"], state["second_momentum_buffer"],
self._muon_momentum_t, self._muon_lr_t, self._muon_wd_t,
self._muon_beta2_t, group["ns_steps"], red_dim)
# Распаковываем обновлённые параметры обратно в список
torch._foreach_copy_(params, list(stacked_params.unbind(0)))
@torch.no_grad()
def step(self):
"""Основной метод шага оптимизатора: диспетчеризация по типам групп."""
for group in self.param_groups:
if group['kind'] == 'adamw':
self._step_adamw(group)
elif group['kind'] == 'muon':
self._step_muon(group)
# ---------------------------------------------------------------------------
# Гиперпараметры (редактировать напрямую, CLI-флаги не нужны)
# ---------------------------------------------------------------------------
# === Архитектура модели ===
ASPECT_RATIO = 64 # model_dim = depth * ASPECT_RATIO (масштабирование ширины)
HEAD_DIM = 128 # Целевая размерность головы внимания
WINDOW_PATTERN = "SSSL" # Паттерн окон: L=полное, S=половинное внимание
# === Оптимизация ===
TOTAL_BATCH_SIZE = 2**19 # ~524K токенов на шаг оптимизатора
EMBEDDING_LR = 0.6 # LR для токен-эмбеддингов (Adam)
UNEMBEDDING_LR = 0.004 # LR для lm_head (Adam)
MATRIX_LR = 0.04 # LR для матричных весов (Muon)
SCALAR_LR = 0.5 # LR для per-layer скаляров (Adam)
WEIGHT_DECAY = 0.2 # "Осторожный" weight decay для Muon
ADAM_BETAS = (0.8, 0.95) # Бета-параметры Adam
WARMUP_RATIO = 0.0 # Доля времени на warmup LR (0 = без warmup)
WARMDOWN_RATIO = 0.5 # Доля времени на cooldown LR
FINAL_LR_FRAC = 0.0 # Финальный LR как доля от начального
# === Размер модели ===
DEPTH = 8 # Количество слоёв трансформера
DEVICE_BATCH_SIZE = 128 # Размер батча на устройство (уменьшить при OOM)
# ---------------------------------------------------------------------------
# Установка: tokenizer, model, optimizer, dataloader
# ---------------------------------------------------------------------------
# === Настройка окружения ===
t_start = time.time()
torch.manual_seed(42)
torch.cuda.manual_seed(42)
torch.set_float32_matmul_precision("high") # Использовать Tensor Cores на GPU
device = torch.device("cuda")
autocast_ctx = torch.amp.autocast(device_type="cuda", dtype=torch.bfloat16)
H100_BF16_PEAK_FLOPS = 989.5e12 # Пиковая производительность H100 в bfloat16
# === Загрузка токенизатора ===
tokenizer = Tokenizer.from_directory()
vocab_size = tokenizer.get_vocab_size()
print(f"Vocab size: {vocab_size:,}")
# === Построение конфигурации модели ===
def build_model_config(depth):
"""Вычисляет d_model и n_head на основе depth и HEAD_DIM."""
base_dim = depth * ASPECT_RATIO
model_dim = ((base_dim + HEAD_DIM - 1) // HEAD_DIM) * HEAD_DIM # Округление вверх до кратного HEAD_DIM
num_heads = model_dim // HEAD_DIM
return GPTConfig(
sequence_len=MAX_SEQ_LEN, vocab_size=vocab_size,
n_layer=depth, n_head=num_heads, n_kv_head=num_heads, n_embd=model_dim,
window_pattern=WINDOW_PATTERN,
)
config = build_model_config(DEPTH)
print(f"Model config: {asdict(config)}")
# === Создание модели на мета-устройстве (экономия памяти) ===
with torch.device("meta"):
model = GPT(config)
model.to_empty(device=device) # Перемещение на GPU без копирования весов
model.init_weights() # Инициализация весов
# === Статистика параметров ===
param_counts = model.num_scaling_params()
print("Parameter counts:")
for key, value in param_counts.items():
print(f" {key:24s}: {value:,}")
num_params = param_counts['total']
num_flops_per_token = model.estimate_flops()
print(f"Estimated FLOPs per token: {num_flops_per_token:e}")
# === Настройка gradient accumulation ===
tokens_per_fwdbwd = DEVICE_BATCH_SIZE * MAX_SEQ_LEN
assert TOTAL_BATCH_SIZE % tokens_per_fwdbwd == 0
grad_accum_steps = TOTAL_BATCH_SIZE // tokens_per_fwdbwd
# === Инициализация оптимизатора ===
optimizer = model.setup_optimizer(
unembedding_lr=UNEMBEDDING_LR,
embedding_lr=EMBEDDING_LR,
scalar_lr=SCALAR_LR,
adam_betas=ADAM_BETAS,
matrix_lr=MATRIX_LR,
weight_decay=WEIGHT_DECAY,
)
# === Компиляция модели через torch.compile ===
model = torch.compile(model, dynamic=False)
# === Подготовка загрузчика данных ===
train_loader = make_dataloader(tokenizer, DEVICE_BATCH_SIZE, MAX_SEQ_LEN, "train")
x, y, epoch = next(train_loader) # Предзагрузка первого батча
print(f"Time budget: {TIME_BUDGET}s")
print(f"Gradient accumulation steps: {grad_accum_steps}")
# ---------------------------------------------------------------------------
# Расписание (все основано на прогрессе = время обучения / TIME_BUDGET)
# ---------------------------------------------------------------------------
def get_lr_multiplier(progress):
"""
Возвращает множитель LR в зависимости от прогресса обучения.
Схема: warmup → plateau → cooldown.
"""
if progress < WARMUP_RATIO:
return progress / WARMUP_RATIO if WARMUP_RATIO > 0 else 1.0
elif progress < 1.0 - WARMDOWN_RATIO:
return 1.0 # Плато
else:
# Линейный спад до FINAL_LR_FRAC
cooldown = (1.0 - progress) / WARMDOWN_RATIO
return cooldown * 1.0 + (1 - cooldown) * FINAL_LR_FRAC
# === Расписание момента для Muon ===
def get_muon_momentum(step):
"""Плавное увеличение момента с 0.85 до 0.95 за первые 300 шагов."""
frac = min(step / 300, 1)
return (1 - frac) * 0.85 + frac * 0.95
# === Расписание weight decay ===
def get_weight_decay(progress):
"""Линейное уменьшение weight decay до 0 к концу обучения."""
return WEIGHT_DECAY * (1 - progress)
# ---------------------------------------------------------------------------
# Цикл обучения
# ---------------------------------------------------------------------------
# === Инициализация цикла ===
t_start_training = time.time()
smooth_train_loss = 0 # EMA для сглаживания потерь
total_training_time = 0
step = 0
while True:
torch.cuda.synchronize() # Синхронизация для точного замера времени
t0 = time.time()
# === Gradient accumulation ===
for micro_step in range(grad_accum_steps):
with autocast_ctx: # Авто-кастинг в bfloat16
loss = model(x, y)
train_loss = loss.detach() # Сохраняем для логирования
loss = loss / grad_accum_steps # Масштабируем для накопления
loss.backward() # Накопление градиентов
x, y, epoch = next(train_loader) # Следующий микро-батч
# === Обновление расписаний ===
progress = min(total_training_time / TIME_BUDGET, 1.0)
lrm = get_lr_multiplier(progress)
muon_momentum = get_muon_momentum(step)
muon_weight_decay = get_weight_decay(progress)
# Применяем расписания к группам оптимизатора
for group in optimizer.param_groups:
group["lr"] = group["initial_lr"] * lrm
if group['kind'] == 'muon':
group["momentum"] = muon_momentum
group["weight_decay"] = muon_weight_decay
# === Шаг оптимизатора ===
optimizer.step()
model.zero_grad(set_to_none=True) # Очистка градиентов
train_loss_f = train_loss.item()
# === Быстрая проверка на взрыв потерь ===
if math.isnan(train_loss_f) or train_loss_f > 100:
print("FAIL")
exit(1)
torch.cuda.synchronize()
t1 = time.time()
dt = t1 - t0 # Время шага
# Считаем время обучения только после первых 10 шагов (исключаем компиляцию)
if step > 10:
total_training_time += dt
# === Логирование ===
ema_beta = 0.9
smooth_train_loss = ema_beta * smooth_train_loss + (1 - ema_beta) * train_loss_f
debiased_smooth_loss = smooth_train_loss / (1 - ema_beta**(step + 1)) # Дебайасинг EMA
pct_done = 100 * progress
tok_per_sec = int(TOTAL_BATCH_SIZE / dt)
mfu = 100 * num_flops_per_token * TOTAL_BATCH_SIZE / dt / H100_BF16_PEAK_FLOPS
remaining = max(0, TIME_BUDGET - total_training_time)
print(f"rstep {step:05d} ({pct_done:.1f}%) | loss: {debiased_smooth_loss:.6f} | lrm: {lrm:.2f} | dt: {dt*1000:.0f}ms | tok/sec: {tok_per_sec:,} | mfu: {mfu:.1f}% | epoch: {epoch} | remaining: {remaining:.0f}s ", end="", flush=True)
# === Управление сборщиком мусора (избегаем пауз ~500мс) ===
if step == 0:
gc.collect()
gc.freeze()
gc.disable()
elif (step + 1) % 5000 == 0:
gc.collect()
step += 1
# === Проверка завершения по времени ===
# Ждём минимум 10 шагов, чтобы не прервать во время компиляции
if step > 10 and total_training_time >= TIME_BUDGET:
break
print() # Перенос строки после последнего лога
# === Подсчёт общего количества токенов ===
total_tokens = step * TOTAL_BATCH_SIZE
# === Финальная валидация ===
model.eval()
with autocast_ctx:
val_bpb = evaluate_bpb(model, tokenizer, DEVICE_BATCH_SIZE)
# === Финальный отчёт ===
t_end = time.time()
startup_time = t_start_training - t_start
steady_state_mfu = 100 * num_flops_per_token * TOTAL_BATCH_SIZE * (step - 10) / total_training_time / H100_BF16_PEAK_FLOPS if total_training_time > 0 else 0
peak_vram_mb = torch.cuda.max_memory_allocated() / 1024 / 1024
print("---")
print(f"val_bpb: {val_bpb:.6f}")
print(f"training_seconds: {total_training_time:.1f}")
print(f"total_seconds: {t_end - t_start:.1f}")
print(f"peak_vram_mb: {peak_vram_mb:.1f}")
print(f"mfu_percent: {steady_state_mfu:.2f}")
print(f"total_tokens_M: {total_tokens / 1e6:.1f}")
print(f"num_steps: {step}")
print(f"num_params_M: {num_params / 1e6:.1f}")
print(f"depth: {DEPTH}")Основные параметры:
Переменная, константа | Значение | Пояснение |
|---|---|---|
|
| Включает динамическое расширение сегментов памяти для уменьшения фрагментации |
|
| Отключает визуальные прогресс-бары при загрузке моделей/данных |
|
| Версия CUDA-архитектуры (например, |
| функция | Интерфейс к оптимизированному ядру flash-attention v3 |
Параметры класса CausalSelfAttention — причинное самовнимание с оптимизациями
Параметр | Назначение |
|---|---|
| Несколько query-голов делят одни и те же KV-головы → экономия памяти и вычислений |
| Обучаемый гейт, который решает, насколько сильно смешивать value-эмбеддинг с обычными value-векторами |
| Оптимизированное ядро, которое вычисляет attention за один проход с подзапросом к памяти |
| Ограничивает внимание токеном только последними W позициями (sliding window) |
Гиперпараметры (редактируются напрямую):
Параметр | Значение | Пояснение |
|---|---|---|
| 64 | Соотношение глубины и ширины: |
| 128 | Размерность одной головы внимания (фиксирована для эффективности) |
|
| Циклический паттерн окон: слои 0,1,2 — короткие окна, слой 3 — полное |
| 524288 | Глобальный размер батча (достигается через gradient accumulation) |
| 0.2 | Применяется только при совпадении знаков градиента и параметра |
| 0.0 | В autoresearch часто начинают с полного LR (быстрый старт) |
Метрики финальной оценки и отчёта:
Метрика | Пояснение |
|---|---|
| Bits Per Byte на валидации — основная метрика качества (меньше = лучше) |
| Model FLOPs Utilization — эффективность использования GPU (цель: >40%) |
| Пиковое потребление видеопамяти |
| Общее количество обработанных токенов в миллионах |
Термины, понимание которых необходимо для изучения train.py:
Термин | Значение |
|---|---|
GQA (Grouped Query Attention) | Оптимизация: несколько query-голов используют общие KV-головы |
RoPE (Rotary Position Embedding) | Позиционные эмбеддинги через вращение векторов в комплексной плоскости |
RMSNorm | Нормализация через среднеквадратичное значение (без смещения) |
Muon | Оптимизатор второго порядка с ортогонализацией градиентов |
Flash Attention 3 | Оптимизированное ядро для causal attention с subquadratic памятью |
MFU (Model FLOPs Utilization) | Доля теоретической производительности GPU, используемая моделью |
BPB (Bits Per Byte) | Метрика качества: биты информации на байт текста (меньше = лучше) |
Softcapping | Ограничение диапазона логитов через |
Cautious WD | Weight decay применяется только при совпадении знаков градиента и параметра |
Обратите внимание на ключевые особенности реализации:
Гибридный оптимизатор: Muon для матриц + AdamW для остального → лучшее качество при высокой скорости
torch.compile + fused kernels: Все шаги оптимизатора скомпилированы в один граф для минимизации накладных расходов
Эффективная упаковка данных: Best-fit packing из
prepare.pyобеспечивает 100% утилизацию контекстаУстойчивость к сбоям: Быстрая проверка на
NaN/взрыв потерь с немедленным завершениемТочный учёт времени: Исключение времени компиляции из расчёта
TIME_BUDGETУправление памятью:
gc.freeze()+gc.disable()после инициализации устраняет паузы сборщика мусора
-
06.01.26 16:06
justineluscacio
Successful Bitcoin Recovery Testimonials in Europe, USA, UK, Asia, Canada, South Africa.
-
06.01.26 16:07
justineluscacio
Successful Bitcoin Recovery Testimonials in Europe, USA, UK, Asia, Canada, South Africa. BITCOIN RECOVERY EXPERT USDT RECOVERY EXPERT ETH RECOVERY EXPERT How do I hire a hacker to help me recover my lost funds? How to Recover Money Lost to an Online Scam How to recover lost or stolen cryptocurrency Best recovery experts for cryptocurrency DuneNectarOnlineExpert was incredibly helpful in recovering my lost Bitcoin. Telegram: T.me/Dunenectarwebexpert WhatApp: +(1252-271-8900)
-
06.01.26 16:14
justineluscacio
I hope this review gets to as many as possible because I am sure it will save many. On March 15, I received a message on Telegram inviting me to join a Cryptocurrency investment group. At first, I thought it was a harmless invitation. I joined and began to see how a specific crypto and forex trader was helping many people and even gifting out Bitcoin to his loyal followers. I was thrilled; many people in the group were reporting their daily successes and how they were earning a substantial amount. I was jealous; I saw people making a lot of money, but not me. I became furious and decided to contact the group's admin, not realizing I had fallen for their trap. That Telegram channel was a hoax, controlled by an organized team of scammers. Within 3 months of being on that platform, I had given out a whopping 11 Bitcoin in total. I was lost when I was logged out of the investment website and blocked from accessing their Telegram channel. I tried all the means I could think of at that moment, all to no avail. It went from days to weeks, and I still couldn't get back my lost Bitcoin. I was on Medium when I came across a review about digital assets recovery companies. There were quite a few of them, but DuneNectarOnlineExpert caught my attention. I copied their email ( Support @ Dunenectaronlineexpert . Com ), filed a report, and my task was attended to with utmost urgency. After back and forth with this team of cybersecurity experts, DuneNectarOnlineExpert successfully traced the Bitcoin to the scammers' wallet, and it was successfully reclaimed. The team of DuneNectarOnlineExpert was incredibly helpful in recovering my lost Bitcoin. Telegram: T.me/Dunenectarwebexpert WhatApp: +(1252-271-8900)
-
06.01.26 17:03
Matt Kegan
Reach out to SolidBlock Forensics if you want to get back your coins from fake crypto investment or your wallet was compromised and all your coins gone. SolidBlock Forensics provide deep ethical analysis and investigation that enables them to trace these schemes, and recover all your funds. Their services are professional and reliable. http://www.solidblockforensics.com
-
09.01.26 17:11
VERONICAFREDDIE809
Earlier this year, I made a mistake that changed everything. I downloaded what I thought was a legitimate trading app I’d found through a Telegram channel. At first, everything looked real until I tried to withdraw. My entire investment vanished into a bot account, and that’s when the truth hit me: I had been scammed. I can’t describe the feeling. It was as if the ground dropped out from under me. I blamed myself. I felt stupid, ashamed, helpless every painful emotion at once. For a while, I couldn’t even talk about it. I thought no one would understand. But then I found someone Agent Jasmine Lopez ([email protected]) ,She didn’t brush me off or judge me. She took my fear seriously. She followed leads I didn’t even know existed, and identified multiple off-chain indicators and wallet clusters linked to the scammer network, she helped me understand what had truly happened behind the scenes. For the first time since everything fell apart, I felt hope. Hearing that other people students, parents, hardworking people had been targeted the same way made me realize I wasn’t alone. What happened to us wasn’t stupidity. It was a coordinated attack. We were prey in a system built to deceive. And somehow, through all the chaos, Agent Jasmine stepped in and shined a light into the darkest moment of my life. I’m still healing from the experience. It changed me. But it also reminded me that even when you think you’re at the end, sometimes a lifeline appears where you least expect it. Contact her at [email protected] WhatsApp at +44 736-644-5035.
-
10.01.26 14:28
justineluscacio
I hope this review gets to as many as possible because I am sure it will save many. On March 15, I received a message on Telegram inviting me to join a Cryptocurrency investment group. At first, I thought it was a harmless invitation. I joined and began to see how a specific crypto and forex trader was helping many people and even gifting out Bitcoin to his loyal followers. I was thrilled; many people in the group were reporting their daily successes and how they were earning a substantial amount. I was jealous; I saw people making a lot of money, but not me. I became furious and decided to contact the group's admin, not realizing I had fallen for their trap. That Telegram channel was a hoax, controlled by an organized team of scammers. Within 3 months of being on that platform, I had given out a whopping 11 Bitcoin in total. I was lost when I was logged out of the investment website and blocked from accessing their Telegram channel. I tried all the means I could think of at that moment, all to no avail. It went from days to weeks, and I still couldn't get back my lost Bitcoin. I was on Medium when I came across a review about digital assets recovery companies. There were quite a few of them, but DuneNectarOnlineExpert caught my attention. I copied their email ( Support @ Dunenectaronlineexpert . Com ), filed a report, and my task was attended to with utmost urgency. After back and forth with this team of cybersecurity experts, DuneNectarOnlineExpert successfully traced the Bitcoin to the scammers' wallet, and it was successfully reclaimed. The team of DuneNectarOnlineExpert was incredibly helpful in recovering my lost Bitcoin. Telegram: T.me/Dunenectarwebexpert WhatApp: +(1252-271-8900)
-
11.01.26 03:16
austinjarvis
I lost £75,000 in a Bitcoin scam. Scammers used deepfake videos to make it seem like Elon Musk himself pitched the deal. These deepfakes swap faces onto real people with AI. They look so real you can't spot the trick at first. They drew me in with talk of fast cash. Double your money in weeks, they said. Videos showed Musk explaining simple steps to join. I bit. Sent my Bitcoin bit by bit. First small amounts worked. Then I dumped my full savings. Boom. Funds vanished. No more contact. My bank account empty. Bills piled up. Loans to cover rent and food. Sleep gone. Stress hit hard. Felt like my whole life crashed. A mate saw my pain. He shared his own story from last year. Same scam type. He pointed me to Sylvester, a recovery agent. Reach him at Yt7cracker@gmail . com. Or WhatsApp + 1 512 577 7957. Or + 44 7428 662701. Sylvester got to work fast. Tracked the blockchain trails. Dealt with exchanges. Fought the scammers' tricks. Weeks later, my Bitcoin came back. Even the profits they promised. Debt lifted. Life back on track. If you're hit, contact him now.
-
12.01.26 16:24
VERONICAFREDDIE809
Earlier this year, I made a mistake that changed everything. I downloaded what I thought was a legitimate trading app I’d found through a Telegram channel. At first, everything looked real until I tried to withdraw. My entire investment vanished into a bot account, and that’s when the truth hit me: I had been scammed. I can’t describe the feeling. It was as if the ground dropped out from under me. I blamed myself. I felt stupid, ashamed, helpless every painful emotion at once. For a while, I couldn’t even talk about it. I thought no one would understand. But then I found someone Agent Jasmine Lopez ([email protected]) ,She didn’t brush me off or judge me. She took my fear seriously. She followed leads I didn’t even know existed, and identified multiple off-chain indicators and wallet clusters linked to the scammer network, she helped me understand what had truly happened behind the scenes. For the first time since everything fell apart, I felt hope. Hearing that other people students, parents, hardworking people had been targeted the same way made me realize I wasn’t alone. What happened to us wasn’t stupidity. It was a coordinated attack. We were prey in a system built to deceive. And somehow, through all the chaos, Agent Jasmine stepped in and shined a light into the darkest moment of my life. I’m still healing from the experience. It changed me. But it also reminded me that even when you think you’re at the end, sometimes a lifeline appears where you least expect it. Contact her at [email protected] WhatsApp at +44 736-644-5035.
-
19.01.26 13:04
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
19.01.26 13:05
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
19.01.26 15:53
VERONICAFREDDIE809
Earlier this year, I made a mistake that changed everything. I downloaded what I thought was a legitimate trading app I’d found through a Telegram channel. At first, everything looked real until I tried to withdraw. My entire investment vanished into a bot account, and that’s when the truth hit me: I had been scammed. I can’t describe the feeling. It was as if the ground dropped out from under me. I blamed myself. I felt stupid, ashamed, helpless every painful emotion at once. For a while, I couldn’t even talk about it. I thought no one would understand. But then I found someone Agent Jasmine Lopez ([email protected]) ,She didn’t brush me off or judge me. She took my fear seriously. She followed leads I didn’t even know existed, and identified multiple off-chain indicators and wallet clusters linked to the scammer network, she helped me understand what had truly happened behind the scenes. For the first time since everything fell apart, I felt hope. Hearing that other people students, parents, hardworking people had been targeted the same way made me realize I wasn’t alone. What happened to us wasn’t stupidity. It was a coordinated attack. We were prey in a system built to deceive. And somehow, through all the chaos, Agent Jasmine stepped in and shined a light into the darkest moment of my life. I’m still healing from the experience. It changed me. But it also reminded me that even when you think you’re at the end, sometimes a lifeline appears where you least expect it. Contact her at [email protected] WhatsApp at +44 736-644-5035.
-
22.01.26 07:48
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
22.01.26 07:50
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
22.01.26 10:42
Tonerdomark
I got my money back from the Elon Musk scam. It cost me over 1 BTC and $55,000 in Dogecoin. Scammers vowed to double investments. Their sites seemed real. Fraud was tough to catch early. They hooked me with fast doubles in weeks or months. Videos showed Musk promising giveaways and gains. I bought in. I sent Bitcoin and Dogecoin in bits at first. Small sends worked. Then I wired my full savings. It vanished quick. No answers came. Bank account empty. Bills piled up. Loans covered rent and food. Sleep fled. Stress hit hard. Life crumbled. A friend spotted my trouble. He told of his scam loss last year. Same old plays. He pointed me to Sylvester Bryant, a recovery expert. Email Yt7cracker@gmail. com. WhatsApp +1 512 577 7957 or +44 7428 662701. Sylvester acted fast. He tracked blockchain trails. Dealt with exchanges. Outsmarted the scammers. In weeks, my Bitcoin came back. Even their phony profits too. Debts gone. Life back on track. Got hit? Contact him now.
-
22.01.26 19:25
Angela_Moore
Help to recover money from elon musk giveaway scam I got my money back from the Elon Musk scam. It cost me over 1 BTC and $55,000 in Dogecoin. Scammers vowed to double investments. Their sites seemed real. Fraud was tough to catch early. They hooked me with fast doubles in weeks or months. Videos showed Musk promising giveaways and gains. I bought in. I sent Bitcoin and Dogecoin in bits at first. Small sends worked. Then I wired my full savings. It vanished quick. No answers came. Bank account empty. Bills piled up. Loans covered rent and food. Sleep fled. Stress hit hard. Life crumbled. A friend spotted my trouble. He told of his scam loss last year. Same old plays. He pointed me to Sylvester Bryant, a recovery expert. Email Yt7cracker@gmail. com. WhatsApp +1 512 577 7957 or +44 7428 662701. Sylvester acted fast. He tracked blockchain trails. Dealt with exchanges. Outsmarted the scammers. In weeks, my Bitcoin came back. Even their phony profits too. Debts gone. Life back on track. Got hit? Contact him now.
-
23.01.26 07:35
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
23.01.26 07:35
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
26.01.26 10:36
alksnismareks
It all started when I decided to explore online trading as a way to grow my savings. Like many, I trusted what appeared to be a legitimate platform, only to find myself trapped in a nightmare. After making consistent trades and finally deciding to withdraw my profits, I was met with silence. My account was suddenly restricted—no warning, no explanation. Every attempt to contact the broker went unanswered or was met with vague, dismissive replies. For three long, agonizing months, I lived in uncertainty. I couldn’t sleep at night. I replayed every email, every transaction, wondering if I’d made a mistake. But deep down, I knew the truth: I hadn’t done anything wrong. The broker had simply decided to lock me out and keep my money. During that time, I felt completely powerless—like I was shouting into a void. The stress affected my health, my relationships, and my ability to focus on anything else. There were days I truly believed that $167,000 was gone forever, lost to the shadows of the unregulated online trading world. I even began to accept it as a painful lesson—one that would cost me dearly but might teach me to be more cautious in the future. But something inside me refused to surrender completely. That’s when I discovered TechY Force Cyber Retrieval. At first, I was cautious—after being scammed once, I didn’t want to fall victim again. But everything about TechY Force felt different. They were transparent from the start. No grand promises, no pressure tactics. Just clear, professional communication and a deep understanding of how these fraudulent brokers operate. Most importantly, they are a licensed specialist in binary options and forex fund recovery, which gave me the confidence to move forward. From our very first consultation, their team treated my case with urgency and empathy. They walked me through the entire process, explained the legal and technical avenues available, and assured me they would handle every detail. They collected documentation, analyzed transaction trails, and engaged directly with the payment processors and the broker using precise, strategic methods I never could have navigated on my own. What happened next was nothing short of miraculous. Within weeks, the broker—who had ignored me for months—began responding. And then, without any further drama or delays, my full $167,000 USD was returned to me. No deductions. No hidden fees. Just clean, complete recovery. The relief I felt was indescribable. It wasn’t just about the money—it was about reclaiming control, restoring trust, and proving that even in the face of deception, there are still good people who fight for what’s right. If you’ve been locked out of your trading account, scammed by a fake investment platform, or had your funds unjustly withheld, please know this: you are not alone, and your money may not be lost forever. Thanks to TechY Force Cyber Retrieval, I got my life back. Their expertise, integrity, and unwavering commitment turned my despair into deliverance. I cannot recommend them highly enough. To anyone reading this in distress: don’t give up. Reach out. Take that step. Because if someone like me—broken, doubtful, and nearly hopeless—can recover every dollar… so can you. WhatsApp them + 156 172 63 697 With heartfelt thanks and renewed hope, — A Recovered and Grateful Client
-
26.01.26 23:21
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
26.01.26 23:21
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
26.01.26 23:21
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.01.26 01:18
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
27.01.26 01:19
Kelvin Alfons
Hello everyone. I’d like to share my personal experience from one of the most challenging times in my life. I’m based in Sydney, Australia, and on November 13, 2025, I fell victim to a fraudulent cryptocurrency investment platform that promised substantial financial growth. Believing their claims, I invested a total of $220,000 with the expectation of earning solid returns. However, when I attempted to withdraw my funds, all communication abruptly stopped. My calls were ignored, my emails went unanswered, and I was left feeling completely powerless. Like many others, I had heard that Bitcoin transactions are impossible to trace, so I assumed my money was lost forever. After some time, I discovered information about GREAT WHIP RECOVERY CYBER SERVICES, a reputable digital asset recovery firm. I decided to reach out to them, and to my astonishment, they were able to help me recover the full amount I had lost. I’m sharing my story in the hope that it may help someone else who is going through a similar situation and looking for support. Their contact is, Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Email:[email protected] Call: +1(406)2729101
-
27.01.26 09:29
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.01.26 09:29
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.01.26 09:32
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
29.01.26 05:03
joyo
The digital world of cryptocurrency offers big chances, but it also hides tricky scams. Losing your crypto to fraud feels awful. It can leave you feeling lost and violated. This guide tells you what to do right away if a crypto scam has hit you. These steps can help you get funds back or stop more trouble. Knowing what to do fast can change everything,reach marie ([email protected] and whatsapp:+1 7127594675)
-
30.01.26 08:23
joseph67t
It's a joy to write this review. Since I began working with Marie at the beginning of 2018, the service has been outstanding. Hackers stole my monies, and I was frightened about how I would get them back. I didn't know where to begin, consequently it was a nightmare for me. But once my friend told me about ([email protected] and whatsap:+1 7127594675), things became simple for me. I'm glad she was able to get my bitcoin back so I could start trading on Binance again!
-
31.01.26 00:55
harristhomas7376
"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
31.01.26 00:55
harristhomas7376
"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
02.02.26 18:52
Christopherbelle
Sylvester Bryant is a top crypto recovery agent! Then I contacted them with my story that i have been scammed. It took time, yet my stolen crypto was recovered . Need help? Reach out to Sylvester on WhatsApp at +1 512 577 7957 or +44 7428 662701. Or email yt7cracker@gmail . com.
-
03.02.26 08:05
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
03.02.26 08:05
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
04.02.26 16:23
borutaralf
GREAT WHIP RECOVERY CYBER SERVICES TRUSTED EXPERTS IN ONLINE RECOVERY SOLUTIONS PHONE CALL:+1(406)2729101 I was unfortunately deceived and scammed out of $88,000 by someone I trusted to manage my funds during a transaction we carried out together. The experience left me deeply disappointed and hurt, realizing that someone could betray that level of trust without any remorse. Determined to seek justice and recover what was stolen, I began searching for legal assistance and came across numerous testimonials about GREAT WHIP RECOVERY CYBER SERVICES, a group known for helping victims recover lost funds. From what I learned, they have successfully assisted many people facing similar situations, returning stolen funds to their rightful owners in a remarkably short time. In my case, the GREAT WHIP RECOVERY CYBER SERVICES were able to recover my funds within just 48 hours, which was truly unbelievable. Even more reassuring was the fact that the scammer was identified, located, and eventually arrested by local authorities in his region. That outcome brought a great sense of relief and closure. I hope this information helps others who have lost their hard-earned money due to misplaced trust. If you’re in a similar situation, you can contact them through their info below to seek help in recovering your stolen funds. Email: [email protected] Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site Phone Call:+1(406)2729101
-
04.02.26 16:24
borutaralf
GREAT WHIP RECOVERY CYBER SERVICES TRUSTED EXPERTS IN ONLINE RECOVERY SOLUTIONS PHONE CALL:+1(406)2729101 I was unfortunately deceived and scammed out of $88,000 by someone I trusted to manage my funds during a transaction we carried out together. The experience left me deeply disappointed and hurt, realizing that someone could betray that level of trust without any remorse. Determined to seek justice and recover what was stolen, I began searching for legal assistance and came across numerous testimonials about GREAT WHIP RECOVERY CYBER SERVICES, a group known for helping victims recover lost funds. From what I learned, they have successfully assisted many people facing similar situations, returning stolen funds to their rightful owners in a remarkably short time. In my case, the GREAT WHIP RECOVERY CYBER SERVICES were able to recover my funds within just 48 hours, which was truly unbelievable. Even more reassuring was the fact that the scammer was identified, located, and eventually arrested by local authorities in his region. That outcome brought a great sense of relief and closure. I hope this information helps others who have lost their hard-earned money due to misplaced trust. If you’re in a similar situation, you can contact them through their info below to seek help in recovering your stolen funds. Email: [email protected] Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site Phone Call:+1(406)2729101
-
04.02.26 17:11
wendytaylor015
My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
05.02.26 12:07
Thomas Muller
YOU CAN REACH OUT TO GREAT WHIP RECOVERY CYBER SERVICES FOR HELP TO RECOVER YOUR STOLEN BTC OR ETH BACK CALL:+1(406)2729101 I once fell victim to online investment scheme that cost me a devastating €254,000. I’m Thomas Muller from Berlin, Germany. The person I trusted turned out to be a fraud, and the moment I realized I’d been deceived, my entire world stopped. I immediately began searching for legitimate ways to recover my funds and hold the scammer accountable. During my search, I came across several testimonies of how Great Whip Recovery Cyber Services helped some people recover money they lost to cyber fraud, I contacted Great Whip Recovery Cyber Service team and provided all the evidence I had. Within about 36 hours, the experts traced the digital trail left by the fraudster, the individual was eventually tracked down and I recovered all my money back. You can contact them with, website https://greatwhiprecoveryc.wixsite.com/greatwhip-site text +1(406)2729101 email [email protected]
-
05.02.26 15:46
feliciabotezatu
Losing access to your cryptocurrency can be devastating—whether you’ve been scammed, hacked, or locked out due to a forgotten password. Many assume their digital assets are gone forever. But with the right expertise, recovery is not only possible—it’s our daily reality. At TECHY FORCE CYBER RETRIEVAL (TFCR), we’re a globally recognized, fully legitimate crypto recovery service dedicated to helping victims reclaim lost or stolen digital assets—safely, ethically, and effectively. Who We Are Backed by a team of certified blockchain forensic analysts, cybersecurity specialists, and ethical hackers, TFCR has recovered millions of dollars in Bitcoin, Ethereum, USDT, and other major cryptocurrencies for clients worldwide. We specialize in cases involving: - Investment scams and fake platforms - Wallet hacks and unauthorized transactions - Forgotten passwords, seed phrases, or corrupted backups - Inaccessible hardware or software wallets Our mission is clear: Help you recover what’s rightfully yours—with honesty, transparency, and proven results. How We Work 1. Confidential Case Review Share your situation with us—no cost, no obligation. We assess whether your case is recoverable based on transaction data, wallet details, and loss type. 2. Advanced Blockchain Forensics Using industry-leading tools, we trace your funds across blockchains, identify destination addresses, and determine if assets are held on exchanges or recoverable platforms—even after complex laundering attempts. 3. Custom Recovery Execution Depending on your case, we: - Reconstruct access to locked wallets using secure decryption methods - Engage with exchanges or payment processors to freeze or retrieve funds - Provide forensic reports to support legal or compliance actions - Negotiate with third parties when appropriate and safe 4. Secure Return & Prevention Advice Recovered assets go directly to a wallet you control. We also offer practical guidance to help you avoid future losses—because security starts after recovery. Why Choose TFCR? No Recovery, No Fee – You only pay upon successful retrieval Legitimate & Transparent – No upfront payments, no hidden costs Global Expertise – Proven success across 50+ countries Ethical Standards – All actions comply with cybersecurity and privacy best practices While crypto threats grow daily, so does our resolve. At TECHY FORCE CYBER RETRIEVAL, we don’t just track transactions—we restore trust, hope, and financial peace of mind. Don’t give up on your crypto. Act now—before critical evidence disappears. 📧 Email: [email protected] 🌐 Visit: Official https://techyforcecyberretrieval.com Website] 🕒 Available 24/7 for urgent cases Your crypto may be missing—but with TFCR, it’s never truly lost. ©️ 2026 TECHY FORCE CYBER RETRIEVAL — Trusted. Professional. Results-Driven.
-
05.02.26 15:52
harryjones5
How Can I Contact a Cryptocurrency Recovery Company? Visit iFORCE HACKER RECOVERY I realize how volatile and thrilling cryptocurrency can be. After joining a Telegram-based service, I made consistent profits for six months before unexpected faults deprived me of approximately $343,000. Withdrawal blunders, little help, and rising dread kept me stuck. I then discovered iForce Hacker Recovery from positive reviews. They replied swiftly, handled my issue professionally, and walked me through every step. My valuables were returned within a week, giving me back my confidence. I heartily recommend their dependable, professional aid services. Contact Info: Website address: htt p:// iforcehackers. co m. Email: iforcehk @ consultant .co m WhatsApp: +1 240 803-3706
-
06.02.26 14:44
feliciabotezatu
Losing access to your cryptocurrency can be devastating—whether you’ve been scammed, hacked, or locked out due to a forgotten password. Many assume their digital assets are gone forever. But with the right expertise, recovery is not only possible—it’s our daily reality. At TECHY FORCE CYBER RETRIEVAL (TFCR), we’re a globally recognized, fully legitimate crypto recovery service dedicated to helping victims reclaim lost or stolen digital assets—safely, ethically, and effectively. Who We Are Backed by a team of certified blockchain forensic analysts, cybersecurity specialists, and ethical hackers, TFCR has recovered millions of dollars in Bitcoin, Ethereum, USDT, and other major cryptocurrencies for clients worldwide. We specialize in cases involving: - Investment scams and fake platforms - Wallet hacks and unauthorized transactions - Forgotten passwords, seed phrases, or corrupted backups - Inaccessible hardware or software wallets Our mission is clear: Help you recover what’s rightfully yours—with honesty, transparency, and proven results. How We Work 1. Confidential Case Review Share your situation with us—no cost, no obligation. We assess whether your case is recoverable based on transaction data, wallet details, and loss type. 2. Advanced Blockchain Forensics Using industry-leading tools, we trace your funds across blockchains, identify destination addresses, and determine if assets are held on exchanges or recoverable platforms—even after complex laundering attempts. 3. Custom Recovery Execution Depending on your case, we: - Reconstruct access to locked wallets using secure decryption methods - Engage with exchanges or payment processors to freeze or retrieve funds - Provide forensic reports to support legal or compliance actions - Negotiate with third parties when appropriate and safe 4. Secure Return & Prevention Advice Recovered assets go directly to a wallet you control. We also offer practical guidance to help you avoid future losses—because security starts after recovery. Why Choose TFCR? No Recovery, No Fee – You only pay upon successful retrieval Legitimate & Transparent – No upfront payments, no hidden costs Global Expertise – Proven success across 50+ countries Ethical Standards – All actions comply with cybersecurity and privacy best practices While crypto threats grow daily, so does our resolve. At TECHY FORCE CYBER RETRIEVAL, we don’t just track transactions—we restore trust, hope, and financial peace of mind. Don’t give up on your crypto. Act now—before critical evidence disappears. Email: [email protected] Visit: Official https://techyforcecyberretrieval.com Website] Available 24/7 for urgent cases Your crypto may be missing—but with TFCR, it’s never truly lost. ©️ 2026 TECHY FORCE CYBER RETRIEVAL — Trusted. Professional. Results-Driven.
-
07.02.26 00:44
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
07.02.26 00:44
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
07.02.26 04:43
Matt Kegan
Reach out to SolidBlock Forensics if you want to get back your coins from fake crypto investment or your wallet was compromised and all your coins gone. SolidBlock Forensics provide deep ethical analysis and investigation that enables them to trace these schemes, and recover all your funds. Their services are professional and reliable. http://www.solidblockforensics.com
-
07.02.26 17:31
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
10.02.26 23:52
frankqq
It is a pleasure to write this review. Since I began working with Marie in early 2018, the service has been outstanding. My coins were stolen by hackers, and I was afraid I wouldn't be able to recover them. It was a nightmare for me because I didn't know where to start. But after my friend told me about [email protected] and whatsapp:+1 7127594675, things became simple for me. I'm glad she was able to get my bitcoin back so I could start trading again.
-
11.02.26 05:50
patricialovick86
How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
11.02.26 05:50
patricialovick86
How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
12.02.26 23:55
brouwerspatrick8
I’ve always believed that sustainability begins at home—not just in how we recycle or conserve energy, but in the very structures we live in. For years, I dreamed of building a zero-waste neighborhood where every house functions like a living ecosystem: solar-powered, water-wise, and crowned with rooftop greenhouses that feed families and filter air. It wasn’t just architecture—it was my vision for a quieter, cleaner future. To make it real, I turned to Bitcoin. Not as a speculative bet, but as a long-term store of value aligned with my values—decentralized, transparent, and independent of broken systems. Over seven years, I poured savings, side income, and relentless discipline into building a $680,000 crypto portfolio. Every coin had a purpose: permits, materials, and community partnerships. My dream had a balance sheet. Then, in one exhausted, distracted moment, it all collapsed. It was November 2025. I was juggling contractor delays, city inspections, and endless design revisions. My nerves were frayed, my coffee pot never empty. When a “Ledger Live Update” notification popped up, I didn’t think twice. The interface looked identical—same logo, same layout. I entered my credentials… and within seconds, the app disappeared. My wallet balance dropped to zero. I sat frozen. My stomach dropped. All that work—years of sacrifice—gone in a blink. The days that followed were dark. I scoured forums, filed reports, and replayed my mistake on loop. Guilt ate at me. How could I have been so careless? My greenhouse renderings sat untouched. My dream felt like a cruel joke. Just when I was ready to walk away, I stumbled upon a newsletter about green innovation. Tucked between articles on carbon-neutral cities and next-gen solar panels was a short feature on *Digital Light Solution*—a specialized team that helps victims of crypto theft recover stolen assets. Skeptical but desperate, I reached out. What followed wasn’t magic—but it was close to it. Their team treated my case with urgency and compassion. They traced the transaction trail, identified the laundering path, and worked with exchanges to freeze what they could. Within weeks, they’d recovered a significant portion of my funds—enough to restart. Today, I’m not just rebuilding my portfolio—I’m breaking ground on my prototype greenhouse. And every beam, every pane of glass, carries the lesson I learned: that even in our most vulnerable moments, there’s still light to be found. [email protected] Telegram ——digitallightsolution website https://digitallightsolution.com/ WHAT'S APP https://wa.link/989vlf
-
12.02.26 23:56
brouwerspatrick8
I’ve always believed that sustainability begins at home—not just in how we recycle or conserve energy, but in the very structures we live in. For years, I dreamed of building a zero-waste neighborhood where every house functions like a living ecosystem: solar-powered, water-wise, and crowned with rooftop greenhouses that feed families and filter air. It wasn’t just architecture—it was my vision for a quieter, cleaner future. To make it real, I turned to Bitcoin. Not as a speculative bet, but as a long-term store of value aligned with my values—decentralized, transparent, and independent of broken systems. Over seven years, I poured savings, side income, and relentless discipline into building a $680,000 crypto portfolio. Every coin had a purpose: permits, materials, and community partnerships. My dream had a balance sheet. Then, in one exhausted, distracted moment, it all collapsed. It was November 2025. I was juggling contractor delays, city inspections, and endless design revisions. My nerves were frayed, my coffee pot never empty. When a “Ledger Live Update” notification popped up, I didn’t think twice. The interface looked identical—same logo, same layout. I entered my credentials… and within seconds, the app disappeared. My wallet balance dropped to zero. I sat frozen. My stomach dropped. All that work—years of sacrifice—gone in a blink. The days that followed were dark. I scoured forums, filed reports, and replayed my mistake on loop. Guilt ate at me. How could I have been so careless? My greenhouse renderings sat untouched. My dream felt like a cruel joke. Just when I was ready to walk away, I stumbled upon a newsletter about green innovation. Tucked between articles on carbon-neutral cities and next-gen solar panels was a short feature on *Digital Light Solution*—a specialized team that helps victims of crypto theft recover stolen assets. Skeptical but desperate, I reached out. What followed wasn’t magic—but it was close to it. Their team treated my case with urgency and compassion. They traced the transaction trail, identified the laundering path, and worked with exchanges to freeze what they could. Within weeks, they’d recovered a significant portion of my funds—enough to restart. Today, I’m not just rebuilding my portfolio—I’m breaking ground on my prototype greenhouse. And every beam, every pane of glass, carries the lesson I learned: that even in our most vulnerable moments, there’s still light to be found. [email protected] Telegram ——digitallightsolution website https://digitallightsolution.com/ WHAT'S APP https://wa.link/989vlf
-
13.02.26 00:17
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
13.02.26 00:17
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
13.02.26 02:16
Ralf Boruta
GREAT WHIP RECOVERY CYBER SERVICES TRUSTED EXPERTS IN ONLINE RECOVERY SOLUTIONS PHONE CALL:+1(406)2729101 I was unfortunately deceived and scammed out of $88,000 by someone I trusted to manage my funds during a transaction we carried out together. The experience left me deeply disappointed and hurt, realizing that someone could betray that level of trust without any remorse. Determined to seek justice and recover what was stolen, I began searching for legal assistance and came across numerous testimonials about GREAT WHIP RECOVERY CYBER SERVICES, a group known for helping victims recover lost funds. From what I learned, they have successfully assisted many people facing similar situations, returning stolen funds to their rightful owners in a remarkably short time. In my case, the GREAT WHIP RECOVERY CYBER SERVICES were able to recover my funds within just 48 hours, which was truly unbelievable. Even more reassuring was the fact that the scammer was identified, located, and eventually arrested by local authorities in his region. That outcome brought a great sense of relief and closure. I hope this information helps others who have lost their hard-earned money due to misplaced trust. If you’re in a similar situation, you can contact them through their info below to seek help in recovering your stolen funds. Email: [email protected] Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site Phone Call:+1(406)2729101
-
13.02.26 02:16
Ralf Boruta
GREAT WHIP RECOVERY CYBER SERVICES TRUSTED EXPERTS IN ONLINE RECOVERY SOLUTIONS PHONE CALL:+1(406)2729101 I was unfortunately deceived and scammed out of $88,000 by someone I trusted to manage my funds during a transaction we carried out together. The experience left me deeply disappointed and hurt, realizing that someone could betray that level of trust without any remorse. Determined to seek justice and recover what was stolen, I began searching for legal assistance and came across numerous testimonials about GREAT WHIP RECOVERY CYBER SERVICES, a group known for helping victims recover lost funds. From what I learned, they have successfully assisted many people facing similar situations, returning stolen funds to their rightful owners in a remarkably short time. In my case, the GREAT WHIP RECOVERY CYBER SERVICES were able to recover my funds within just 48 hours, which was truly unbelievable. Even more reassuring was the fact that the scammer was identified, located, and eventually arrested by local authorities in his region. That outcome brought a great sense of relief and closure. I hope this information helps others who have lost their hard-earned money due to misplaced trust. If you’re in a similar situation, you can contact them through their info below to seek help in recovering your stolen funds. Email: [email protected] Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site Phone Call:+1(406)2729101
-
13.02.26 18:29
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
13.02.26 18:29
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
17.02.26 23:59
Lilyfox
These group of CYBER GURUS below helped my family in recovering stolen bitcoin by scammers and they also helped me in securing a university title in one of the best university in the world I'm saying a very big thank you to them contact them now ; [email protected] or WhatsApp +447476606228 -Recovery of funds from fake platform/BINARY TRADING - Retrieval of fraudulent funds - Bank Transfer service - BITCOIN TOP UP - Money, recovery from any country in the world - Change of university degrees - Spying of all social media account within - Sales of Blank ATM and Credit Cards - Sales of university Titles originals. - Clearing of bank debts - University title offer and so many others ... Despite all odds these internet gurus have proven themselves worthy to be called a professional Cyber genius ... once again i beat up my chest to confess that these group of cyber gurus are reliable and satisfactory with 100% reliability.....
-
18.02.26 00:01
Lilyfox
GENERAL HACKING AND CRYPTO RECOVERY SERVICES These group of CYBER GURUS below helped my family in recovering stolen bitcoin worth of $168,000 USD by scammers and they also helped me in securing a university title in one of the best university in the world I'm saying a very big thank you to them contact them now ; [email protected] or WhatsApp +447476606228 -Recovery of funds from fake platform/BINARY TRADING - Retrieval of fraudulent funds - Bank Transfer service - BITCOIN TOP UP - Money, recovery from any country in the world - Change of university degrees - Spying of all social media account within - Sales of Blank ATM and Credit Cards - Sales of university Titles originals. - Clearing of bank debts - University title offer and so many others ... Despite all odds these internet gurus have proven themselves worthy to be called a professional Cyber genius ... once again i beat up my chest to confess that these group of cyber gurus are reliable and satisfactory with 100% reliability.....
-
18.02.26 03:23
walterlindahi9
This past January, my world came crashing down. I lost nearly $42,000 of my hard-earned savings to a sophisticated Solana-based crypto scam. At first, it all seemed legitimate: sleek website, professional whitepaper, even glowing testimonials from “investors.” I’d done my homework, or so I thought. The promise of high returns in a volatile market felt like my ticket to financial freedom. For the first few months, everything appeared to be working. My portfolio showed steady gains. I remember checking my wallet balance daily, feeling a mix of pride and relief. I’ve cracked the code to building real wealth. Then, without warning, the platform vanished. Wallet addresses went dead. Support channels disappeared, and my funds were gone in an instant. The emotional fallout was worse than the financial loss. Sleepless nights became the norm. Anxiety gnawed at me constantly. I replayed every decision in my head, blaming myself for being naive. I vowed never to trust anyone again, not influencers, not experts, not even my own judgment. But giving up wasn’t an option. I owed it to myself and to my future to fight back. So I began digging. I scoured Reddit threads, filed reports with blockchain analytics firms, and even contacted local authorities (though they offered little help). The more I searched, the more overwhelmed I became, lost in a labyrinth of technical jargon, dead ends, and predatory recovery services asking for upfront fees. Then, through a survivor’s forum, I stumbled upon TechY Force Cyber Retrieval. Skeptical but desperate, I reached out. What set them apart wasn’t just their expertise; it was their empathy. They didn’t make wild promises. Instead, they walked me through how crypto tracing works, what success looks like, and what realistic timelines are. No pressure. No false hope. Within weeks, their forensic team identified transaction trails linked to the scam wallet. Using on-chain analysis and coordination with exchanges, they flagged suspicious activity and initiated recovery protocols. It wasn’t magic, but it was methodical, transparent, and grounded in real blockchain intelligence. Today, I’m cautiously optimistic. While not all funds have been recovered yet, TechY Force has already secured a significant portion and, more importantly, restored my sense of agency. I’m sleeping again. I’m healing. If you’ve been scammed, know this: you’re not alone, and you’re not foolish. Crypto fraud preys on hope, but that same hope can fuel your comeback. Don’t suffer in silence. Reach out. Ask questions. And never let a scammer steal your future along with your funds. WhatsApp +1(561) 726 3697 Mail. Techyforcecyberretrieval(@)consultant(.)com Telegram (@)TechCyberforc
-
22.02.26 03:48
harristhomas7376
"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
22.02.26 03:49
harristhomas7376
"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
22.02.26 18:58
Natasha Williams
I am Natasha Williams from Dallas. I want to share my testimony to encourage anyone who has ever fallen victim to a scam or fraud. Some time ago, I was defrauded by some fraudulent cryptocurrency investment organization online, I was a victim and I lost a huge amount of money, $382,000. I felt angry, disappointed and helpless but I refused to give up and stay calm. I came across this agency, GREAT WHIP RECOVERY CYBER SERVICES.. who helped people recover their money from scammers and the testimonies I saw were quite amazing. And I decided to contact them. I gathered every piece of evidence, chats, receipts, account details, and messages and reported the case to the agency, GREAT WHIP RECOVERY CYBER SERVICES. After 73hours of follow up and not losing faith, the fraudster was traced and held accountable and I recovered all my money back. I highly recommend, GREAT WHIP RECOVERY CYBER SERVICES agency if you have ever fallen victim to scammers, you can contact them. Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site email: [email protected] Call Line: +1(406)2729101
-
22.02.26 19:00
Natasha Williams
I am Natasha Williams from Dallas. I want to share my testimony to encourage anyone who has ever fallen victim to a scam or fraud. Some time ago, I was defrauded by some fraudulent cryptocurrency investment organization online, I was a victim and I lost a huge amount of money, $382,000. I felt angry, disappointed and helpless but I refused to give up and stay calm. I came across this agency, GREAT WHIP RECOVERY CYBER SERVICES.. who helped people recover their money from scammers and the testimonies I saw were quite amazing. And I decided to contact them. I gathered every piece of evidence, chats, receipts, account details, and messages and reported the case to the agency, GREAT WHIP RECOVERY CYBER SERVICES. After 73hours of follow up and not losing faith, the fraudster was traced and held accountable and I recovered all my money back. I highly recommend, GREAT WHIP RECOVERY CYBER SERVICES agency if you have ever fallen victim to scammers, you can contact them. Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site email: [email protected] Call Line: +1(406)2729101
-
23.02.26 23:26
chongfook
As cryptocurrencies continue to reshape global finance in 2026, the risks have never been higher. From sophisticated phishing campaigns to fake wallet apps and investment scams, millions of investors face the devastating reality of lost or stolen digital assets. When your crypto vanishes, panic sets in—and that's when fraudsters strike again, posing as "recovery experts" to exploit your vulnerability. CONTACTS US Techyforcecyberretrieval(@)consultant(.)com https(://)techyforcecyberretrieval(.)com But there's a legitimate path forward. TECHY FORCE CYBER RETRIEVAL stands as the industry's most trusted crypto recovery company, combining advanced blockchain forensics, global partnerships, and a client-centric approach to help victims reclaim what was stolen. --- Why Recovery Is Possible—With the Right Team Cryptocurrency's decentralized, pseudonymous nature makes asset recovery complex—but not impossible. The blockchain is transparent. Every transaction leaves a trail. The challenge isn't finding the funds—it's having the expertise to follow that trail through mixers, bridges, and exchange deposits before they disappear forever. That's where TECHY FORCE CYBER RETRIEVAL excels. --- Our Proven Recovery Framework We don't believe in shortcuts, false promises, or upfront fees. Our process is built on transparency, forensic precision, and real results. Here's how we work: 1. Case Intake & Initial Assessment You begin by submitting a detailed report: compromised wallet addresses, transaction IDs, timestamps, and any communication with scammers. Our intake team reviews your case within hours to determine immediate next steps. 2. Blockchain Forensic Analysis Our specialists deploy proprietary tracking tools to map the movement of your stolen assets across multiple blockchains. We identify laundering patterns, exchange deposit addresses, and potential freezing points—building a clear investigative roadmap. 3. Global Partner Coordination Through established relationships with regulated exchanges, DeFi protocols, and compliance teams worldwide, we initiate direct communication to flag suspicious transactions and request asset freezes where legally permissible. 4. Legal & Regulatory Engagement When necessary, we collaborate with legal partners and law enforcement agencies to strengthen recovery efforts—especially in cases involving large-scale hacks or organized fraud rings. 5. Recovery Execution & Fund Return Once assets are secured, they're transferred directly to a new, secure wallet of your choice. We never hold your funds. And critically, we operate on a success-only model. You pay nothing unless we recover your assets. 6. Post-Recovery Security Guidance Recovery is only half the battle. We provide personalized recommendations to secure your remaining holdings—from hardware wallet setup to phishing awareness training—so you can move forward with confidence. --- What Sets TECHY FORCE CYBER RETRIEVAL Apart While countless "recovery services" flood the internet, few deliver legitimate results. Here's why we're consistently rated the best crypto recovery company in 2026: - Zero Upfront Fees – We only succeed when you do. No hidden charges. No bait-and-switch tactics. - Advanced Blockchain Intelligence – Our forensic tools track assets across Bitcoin, Ethereum, Solana, and 50+ other networks. - Global Reach – Partnerships with exchanges and regulatory bodies in North America, Europe, and Asia maximize recovery odds. - Client-First Communication – Weekly updates. Clear timelines. No ghosting. - Proven Track Record – Hundreds of successful recoveries in 2025–2026, with millions returned to rightful owners. --- Emerging Trends in 2026: What Victims Need to Know The threat landscape evolves constantly. This year's biggest risks include: - AI-Powered Phishing: Scammers now use deepfake voice and video to impersonate support staff. - Cross-Chain Bridge Exploits: Funds moved between networks are increasingly targeted. - Fake Recovery Services: Fraudsters pose as legitimate firms—always verify credentials before sharing information. TECHY FORCE CYBER RETRIEVAL stays ahead of these threats, continuously updating our tools and strategies to protect and serve our clients. CONTACTS US Techyforcecyberretrieval(@)consultant(.)com https(://)techyforcecyberretrieval(.)com --- Your Next Step If you've lost crypto to a scam, hack, or forgotten credentials, don't let despair—or another fraudster—steal your second chance. TECHY FORCE CYBER RETRIEVAL is accessible, transparent, and ready to help. Reach out today. Let our experts assess your case—and show you that even in 2026, stolen crypto doesn't have to stay lost forever. — TECHY FORCE CYBER RETRIEVAL Advanced Forensics. Global Reach. Your Recovery.
-
24.02.26 15:31
[email protected]`
Like many others, I was drawn in by the allure of cryptocurrency and the promise of financial freedom. When I encountered a self-proclaimed "crypto guru" online, his confidence and flashy lifestyle convinced me that he held the key to success. Eager to learn, I parted with $15,000 for his exclusive course, believing it would grant me access to an elite trading group and lucrative market insights. Initially, my excitement was palpable; I truly thought I was on the verge of a breakthrough. However, that enthusiasm quickly curdled into dread. Once inside the group, the dynamic shifted from education to aggressive exploitation. Instead of genuine mentorship, members were relentlessly upsold on fake trading signals that yielded nothing but losses. The pressure escalated when we were encouraged to invest in a supposed "private pool," which required an additional, staggering access fee of $60,000. It was only as I began to notice glaring inconsistencies and a complete lack of real results among the members that the fog lifted. I realized I hadn't joined a community of traders; I had walked into a sophisticated trap designed specifically to prey on newcomers like myself. The realization that the promises of wealth and insider knowledge were nothing more than a façade left me feeling vulnerable, deceived, and financially devastated. The dream of easy returns had turned into a heavy burden of regret. Desperate for a solution and refusing to let the fraudsters win, I began searching for help. That is when I discovered DIGITAL LIGHT SOLUTION, a firm specializing in online fraud investigations. Reaching out to them was the turning point. Their team approached my case with professionalism and empathy, immediately understanding the complexity of the scam. They guided me through the investigation process, uncovering the layers of deception used by the "guru" and his network. Thanks to their expertise and relentless pursuit of justice, I was able to navigate the aftermath of this ordeal with clarity rather than confusion. While the experience was a harsh lesson, connecting with DIGITAL LIGHT SOLUTION restored my hope and proved that there are still allies ready to fight against online exploitation. If you find yourself in a similar situation, do not lose hope—seek professional help immediately. Contact them directly Website https://digitallightsolution.com/ Email — Digitallightsolution(At)qualityservice(DOT)com What's App — https://wa.link/989vlf
-
24.02.26 15:32
[email protected]`
Like many others, I was drawn in by the allure of cryptocurrency and the promise of financial freedom. When I encountered a self-proclaimed "crypto guru" online, his confidence and flashy lifestyle convinced me that he held the key to success. Eager to learn, I parted with $15,000 for his exclusive course, believing it would grant me access to an elite trading group and lucrative market insights. Initially, my excitement was palpable; I truly thought I was on the verge of a breakthrough. However, that enthusiasm quickly curdled into dread. Once inside the group, the dynamic shifted from education to aggressive exploitation. Instead of genuine mentorship, members were relentlessly upsold on fake trading signals that yielded nothing but losses. The pressure escalated when we were encouraged to invest in a supposed "private pool," which required an additional, staggering access fee of $60,000. It was only as I began to notice glaring inconsistencies and a complete lack of real results among the members that the fog lifted. I realized I hadn't joined a community of traders; I had walked into a sophisticated trap designed specifically to prey on newcomers like myself. The realization that the promises of wealth and insider knowledge were nothing more than a façade left me feeling vulnerable, deceived, and financially devastated. The dream of easy returns had turned into a heavy burden of regret. Desperate for a solution and refusing to let the fraudsters win, I began searching for help. That is when I discovered DIGITAL LIGHT SOLUTION, a firm specializing in online fraud investigations. Reaching out to them was the turning point. Their team approached my case with professionalism and empathy, immediately understanding the complexity of the scam. They guided me through the investigation process, uncovering the layers of deception used by the "guru" and his network. Thanks to their expertise and relentless pursuit of justice, I was able to navigate the aftermath of this ordeal with clarity rather than confusion. While the experience was a harsh lesson, connecting with DIGITAL LIGHT SOLUTION restored my hope and proved that there are still allies ready to fight against online exploitation. If you find yourself in a similar situation, do not lose hope—seek professional help immediately. Contact them directly Website https://digitallightsolution.com/ Email — Digitallightsolution(At)qualityservice(DOT)com What's App — https://wa.link/989vlf
-
26.02.26 16:29
michaeldavenport238
I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
26.02.26 16:29
michaeldavenport238
I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.02.26 00:08
sanayoliver
I spend my days studying the mysteries of the universe, delving into black holes, quantum mechanics, and the nature of time itself. But apparently, the real black hole I should have been concerned about was my own memory. I encrypted my Bitcoin wallet to keep it as secure as possible. The problem? I promptly forgot the password. Classic, right? It didn't help that this wasn't just pocket change I was dealing with. No, I had $190,000 in Bitcoin sitting in that wallet, and my mind had decided to take a vacation, leaving me with absolutely no idea what that password was. The panic set in fast. My brain, which could solve some of the most complex physics equations, couldn't remember a 12-character password. It felt like my entire financial future was being sucked into a black hole, one I'd created myself. Desperate, I tried everything. I thought I could outsmart the system, using every trick I could think of. I tried variations of passwords I thought I might have used, analyzing them through the lens of my own behavioral patterns. I even resorted to good ol' brute force, typing random combinations for hours, hoping that maybe, just maybe, my subconscious would strike gold. Spoiler alert: it didn't. Each failed attempt made me feel more and more like a genius who'd locked themselves out of their own universe. In a final act of desperation, admitting that theoretical physics couldn't crack my own encryption, I contacted TechY Force Cyber Retrieval. From the moment I reached out, the difference was night and day. While I had been flailing in the dark, they approached my case with a precision that rivaled the calculations I do daily. They didn't promise miracles; they promised a methodical, advanced recovery process. Within a surprisingly short timeframe, they utilized specialized tools to bypass the mental block I couldn't overcome. When they finally recovered the wallet and confirmed the full $190,000 was intact and accessible, the relief was indescribable. It was as if I had pulled my financial future back from the event horizon just before it was lost forever. To anyone thinking they are too smart to lose their keys, or too logical to make such a mistake: don't wait until you are staring into the abyss. If you find yourself in a situation where your own memory has become your greatest enemy, trust the experts at TechY Force Cyber Retrieval. They turned my personal black hole into a success story, proving that sometimes, even the brightest minds need a little help to find the light. REACH OUT TO THEM ON MAIL [email protected]
-
27.02.26 02:04
elizabethmadison
My name is Elizabeth Madison currently living in New York. There was a time I felt completely broken. I had trusted a fraudulent bitcoin investment organization, who turned out to be a fraudster. I sent money, believing their sweet words and promises on the interest rate I will get back in return, only to realize later that I’ve been scammed. On the day of withdrawal there was no money in my account. The pain hit deep. I couldn’t sleep, I kept asking myself how I could have been so careless, meanwhile my mom was battling with a stroke and the expenses were too much. For days, I cried and blamed myself. The betrayal, the disappointment and my mom's health issues all of this stress made me want to give up on life. But one day, I decided that sitting in pain wouldn’t solve anything. I picked myself up and chose to fight for what I lost then I came across GREAT WHIP RECOVERY CYBER SERVICES and saw how he helped people recover their funds from online fraud. I emailed all the transactions and paperwork I had with the fraudulent organization and they helped me recover all my lost money in just five days. If you have ever fallen victim to scammers, contact GREAT WHIP RECOVERY CYBER SERVICES to help you recover every penny you have lost. (Text +1(406)2729101) (Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site) (Email [email protected])
-
27.02.26 02:04
elizabethmadison
My name is Elizabeth Madison currently living in New York. There was a time I felt completely broken. I had trusted a fraudulent bitcoin investment organization, who turned out to be a fraudster. I sent money, believing their sweet words and promises on the interest rate I will get back in return, only to realize later that I’ve been scammed. On the day of withdrawal there was no money in my account. The pain hit deep. I couldn’t sleep, I kept asking myself how I could have been so careless, meanwhile my mom was battling with a stroke and the expenses were too much. For days, I cried and blamed myself. The betrayal, the disappointment and my mom's health issues all of this stress made me want to give up on life. But one day, I decided that sitting in pain wouldn’t solve anything. I picked myself up and chose to fight for what I lost then I came across GREAT WHIP RECOVERY CYBER SERVICES and saw how he helped people recover their funds from online fraud. I emailed all the transactions and paperwork I had with the fraudulent organization and they helped me recover all my lost money in just five days. If you have ever fallen victim to scammers, contact GREAT WHIP RECOVERY CYBER SERVICES to help you recover every penny you have lost. (Text +1(406)2729101) (Website https://greatwhiprecoveryc.wixsite.com/greatwhip-site) (Email [email protected])
-
27.02.26 15:57
luciajessy3
Don’t be deceived by different testimonies online that is most likely wrong. I have made use of several recovery options that got me disappointed at the end of the day but I must confess that the tech genius I eventually found is the best out here. It’s better you devise your time to find the valid professional that can help you recover your stolen or lost crypto such as bitcoins rather than falling victim of other amateur hackers that cannot get the job done. ADAMWILSON . TRADING @ CONSULTANT COM / is the most reliable and authentic blockchain tech expert you can work with to recover what you lost to scammers. They helped me get back on my feet and I’m very grateful for that. Contact their email today to recover your lost coins ASAP…
-
27.02.26 15:59
wendytaylor015
My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.02.26 15:59
wendytaylor015
My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.02.26 16:00
wendytaylor015
My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
27.02.26 16:01
luciajessy3
Don’t be deceived by different testimonies online that is most likely wrong. I have made use of several recovery options that got me disappointed at the end of the day but I must confess that the tech genius I eventually found is the best out here. It’s better you devise your time to find the valid professional that can help you recover your stolen or lost crypto such as bitcoins rather than falling victim of other amateur hackers that cannot get the job done. ADAMWILSON . TRADING @ CONSULTANT COM / is the most reliable and authentic blockchain tech expert you can work with to recover what you lost to scammers. They helped me get back on my feet and I’m very grateful for that. Contact their email today to recover your lost coins ASAP…
-
27.02.26 16:01
luciajessy3
Don’t be deceived by different testimonies online that is most likely wrong. I have made use of several recovery options that got me disappointed at the end of the day but I must confess that the tech genius I eventually found is the best out here. It’s better you devise your time to find the valid professional that can help you recover your stolen or lost crypto such as bitcoins rather than falling victim of other amateur hackers that cannot get the job done. ADAMWILSON . TRADING @ CONSULTANT COM / is the most reliable and authentic blockchain tech expert you can work with to recover what you lost to scammers. They helped me get back on my feet and I’m very grateful for that. Contact their email today to recover your lost coins ASAP…
-
27.02.26 16:01
luciajessy3
Don’t be deceived by different testimonies online that is most likely wrong. I have made use of several recovery options that got me disappointed at the end of the day but I must confess that the tech genius I eventually found is the best out here. It’s better you devise your time to find the valid professional that can help you recover your stolen or lost crypto such as bitcoins rather than falling victim of other amateur hackers that cannot get the job done. ADAMWILSON . TRADING @ CONSULTANT COM / is the most reliable and authentic blockchain tech expert you can work with to recover what you lost to scammers. They helped me get back on my feet and I’m very grateful for that. Contact their email today to recover your lost coins ASAP…
-
01.03.26 10:48
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
01.03.26 10:48
marcushenderson624
Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
03.03.26 14:09
Thomas Muller
YOU CAN REACH OUT TO GREAT WHIP RECOVERY CYBER SERVICES FOR HELP TO RECOVER YOUR STOLEN BTC OR ETH BACK CALL:+1(406)2729101 I once fell victim to online investment scheme that cost me a devastating €254,000. I’m Thomas Muller from Berlin, Germany. The person I trusted turned out to be a fraud, and the moment I realized I’d been deceived, my entire world stopped. I immediately began searching for legitimate ways to recover my funds and hold the scammer accountable. During my search, I came across several testimonies of how Great Whip Recovery Cyber Services helped some people recover money they lost to cyber fraud, I contacted Great Whip Recovery Cyber Service team and provided all the evidence I had. Within about 36 hours, the experts traced the digital trail left by the fraudster, the individual was eventually tracked down and I recovered all my money back. You can contact them with, website https://greatwhiprecoveryc.wixsite.com/greatwhip-site text +1(406)2729101 email [email protected]
-
03.03.26 14:09
Thomas Muller
YOU CAN REACH OUT TO GREAT WHIP RECOVERY CYBER SERVICES FOR HELP TO RECOVER YOUR STOLEN BTC OR ETH BACK CALL:+1(406)2729101 I once fell victim to online investment scheme that cost me a devastating €254,000. I’m Thomas Muller from Berlin, Germany. The person I trusted turned out to be a fraud, and the moment I realized I’d been deceived, my entire world stopped. I immediately began searching for legitimate ways to recover my funds and hold the scammer accountable. During my search, I came across several testimonies of how Great Whip Recovery Cyber Services helped some people recover money they lost to cyber fraud, I contacted Great Whip Recovery Cyber Service team and provided all the evidence I had. Within about 36 hours, the experts traced the digital trail left by the fraudster, the individual was eventually tracked down and I recovered all my money back. You can contact them with, website https://greatwhiprecoveryc.wixsite.com/greatwhip-site text +1(406)2729101 email [email protected]
-
04.03.26 07:21
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
04.03.26 07:22
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
04.03.26 12:25
patricialovick86
How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
04.03.26 12:25
patricialovick86
How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
06.03.26 13:36
CARL9090
In January, my life shifted in a way I never expected. I clicked a trading link given to me by someone I found on Telegram, believing it was legitimate. It looked professional. It felt secure. I trusted it. Until I tried to withdraw my money. Within seconds, everything was gone, transferred into a wallet claiming account without a trace. That was the moment the truth hit me: I had been scammed. The emotional fallout was brutal. For weeks, I couldn’t even speak about it. I thought people would judge me. I thought they’d say I should have known better. Then someone stepped in who changed everything Agent Jasmine Lopez ,She listened without judgment. She treated my fear as real and valid. She traced patterns, uncovered off-chain indicators, and identified wallet clusters linked to a larger scam network. She showed me that what happened wasn’t random it was organized and intentional. For the first time, I felt hope. Hearing that students, parents, and hardworking people had been targeted the same way made me realize this wasn’t stupidity. It was predation. We weren’t careless we were deliberately targeted and manipulated I’m still healing. The experience changed me. But it also reminded me that even in your darkest moment, there can be someone willing to shine a light. Contact her at [email protected] WHATSAPP +44 7478077894
-
07.03.26 07:46
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
07.03.26 07:46
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
07.03.26 08:39
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
07.03.26 08:55
Jane4
I lost about $600k Bitcoin last year, I searched around and tried to work with some recovery firm unfortunately I was scammed as well. This happened for months until I came across [email protected] They came to my rescue and all my funds were recovered within few days I'm so happy right now .
-
07.03.26 09:40
Alena76
Most people have been scammed severally and they give up on their funds I'm saying these because I was a victim too After loosing 745,000 USD I lose my mind until I read about COIN HACK RECOVERY I decided to contact the company on: [email protected] and I'm glad I made the decision not to give up. they helped me to recover all my lost funds within two days.
-
07.03.26 10:37
Alena76
Most people have been scammed severally and they give up on their funds I'm saying these because I was a victim too After loosing 745,000 USD I lose my mind until I read about COIN HACK RECOVERY I decided to contact the company on: [email protected] and I'm glad I made the decision not to give up. they helped me to recover all my lost funds within two days.
-
07.03.26 10:37
Alena76
Most people have been scammed severally and they give up on their funds I'm saying these because I was a victim too After loosing 745,000 USD I lose my mind until I read about COIN HACK RECOVERY I decided to contact the company on: [email protected] and I'm glad I made the decision not to give up. they helped me to recover all my lost funds within two days.
-
07.03.26 17:49
Natasha Williams
I am Natasha Williams from Dallas. I want to share my testimony to encourage anyone who has ever fallen victim to a scam or fraud. Some time ago, I was defrauded by some fraudulent cryptocurrency investment organization online, I was a victim and I lost a huge amount of money, $382,000. I felt angry, disappointed and helpless but I refused to give up and stay calm. I came across this agency, GREAT WHIP RECOVERY CYBER SERVICES.. who helped people recover their money from scammers and the testimonies I saw were quite amazing. And I decided to contact them. I gathered every piece of evidence, chats, receipts, account details, and messages and reported the case to the agency, GREAT WHIP RECOVERY CYBER SERVICES. After 73hours of follow up and not losing faith, the fraudster was traced and held accountable and I recovered all my money back. I highly recommend, GREAT WHIP RECOVERY CYBER SERVICES agency if you have ever fallen victim to scammers, you can contact them. Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site email: [email protected] Call Line: +1(406)2729101
-
07.03.26 20:10
ericbank61
I never thought I’d be the one writing one of these stories. You hear about crypto scams, hacks, and lost fortunes, and you think, “That’s for other people. The careless ones.” I was careful. Or so I believed. It started with a sophisticated phishing attack. An email that looked identical to a legitimate exchange notification, a link to “verify my wallet security,” and a moment of distracted panic. I clicked. Within hours, my life savings in Bitcoin—a sum I’d been accumulating for five years—vanished from my private wallet. The transaction hash was a cold, unfeeling tombstone on the blockchain. My stomach dropped into a void. I felt physically ill. The police filed a report, but their knowledge ended at the edge of traditional finance. The exchange offered sympathy but no solutions. I was adrift, utterly hopeless. After weeks of despair, scouring forums in the dead of night, I found a thread mentioning Mighty Hacker Recovery. The name sounded almost too bold, like something from a cheesy movie. But the testimonials were detailed, sober, and from people who sounded just like me: desperate, betrayed, and out of options. With nothing left to lose, I reached out. Their intake process was professional but guarded. They asked for transaction IDs, wallet addresses, and a detailed timeline—no promises, just facts. A consultant named Leo became my point of contact. He had a calm, analytical voice that cut through my panic. “We don’t hack *into* systems,” he explained. “We follow the digital trail. We analyze the attack vector, trace the flow of funds through the blockchain’s transparency, and identify the weak points in the scammer’s own security. Sometimes, it’s about speed and outmaneuvering them before they can launder the assets.” What followed was a tense, silent partnership. I provided every shred of information I had, while Leo’s team worked in the shadows. There were days of silence that felt like years. Then, an update: they’d traced my BTC to a mixing service, a tool scammers use to obfuscate the trail. Mighty Hacker Recovery used advanced blockchain forensic techniques to peel back those layers. They discovered the scammer had made a critical error—a small portion of the funds was sent to a KYC-compliant exchange wallet. That was the chink in the armor. Using the immutable evidence from the blockchain and legal pressure channels they’d established with certain international platforms, they initiated a recovery claim. The process was complex, involving digital affidavits and proof of illicit origin. Three weeks after my first desperate email, Leo called. “We’ve secured a freeze on the destination wallet. The exchange is cooperating. We’re initiating the reversal.” I didn’t dare believe it until I saw it. Two days later, my wallet balance updated. My Bitcoin, minus Mighty Hacker Recovery’s contingency fee, was back. The relief wasn’t euphoric; it was a deep, trembling exhaustion, like waking up from a nightmare. They didn’t perform magic. They applied intense expertise, relentless persistence, and an intricate understanding of both the blockchain’s weaknesses and a scammer’s psychology. They gave me back more than my crypto; they gave me back a sense of agency in a landscape designed to make victims feel powerless. If you’re reading this from your own private hell of loss, know this: the trail never truly disappears. You just need the right team to follow it. For me, that was Mighty Hacker Recovery.
-
07.03.26 22:44
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
07.03.26 22:44
robertalfred175
CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
-
11.03.26 19:43
Michael Jensen
With the help and expertise of CapitalNode Analytics, i was able to get back my digital tokens from a fake investment platform. They are swift, precise and transparent in their operations.
-
12.03.26 15:04
Mike Franz
I recently ran into a serious issue with my cryptocurrency account that left me unable to access my bitcoin wallet. After several failed login attempts and repeated blocks from the system, I began to worry that I might lose access to my $415,000 permanently. Determined to fix the problem, After spending hours reading a review of GREAT WHIP RECOVERY CYBER SERVICES, on how they successfully assisted countless individuals in similar situations as mine. The process was stressful, but eventually the issue was resolved and I was able to regain access to my bitcoin wallet account. I’m immensely grateful to GREAT WHIP RECOVERY CYBER SERVICES for their incredible work, for those who need help, you can contact through the following channels: Phone Call: +1(406)2729101 Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Mail: [email protected]
-
12.03.26 15:05
Mike Franz
I recently ran into a serious issue with my cryptocurrency account that left me unable to access my bitcoin wallet. After several failed login attempts and repeated blocks from the system, I began to worry that I might lose access to my $415,000 permanently. Determined to fix the problem, After spending hours reading a review of GREAT WHIP RECOVERY CYBER SERVICES, on how they successfully assisted countless individuals in similar situations as mine. The process was stressful, but eventually the issue was resolved and I was able to regain access to my bitcoin wallet account. I’m immensely grateful to GREAT WHIP RECOVERY CYBER SERVICES for their incredible work, for those who need help, you can contact through the following channels: Phone Call: +1(406)2729101 Website: https://greatwhiprecoveryc.wixsite.com/greatwhip-site Mail: [email protected]
Для участия в Чате вам необходим бесплатный аккаунт pro-blockchain.com
Войти
Регистрация
Похожие новости
Подпишись на telegram канал и узнавай первым обо всех новостях