Этот сайт использует файлы cookies. Продолжая просмотр страниц сайта, вы соглашаетесь с использованием файлов cookies. Если вам нужна дополнительная информация, пожалуйста, посетите страницу Политика файлов Cookie
Subscribe
Прямой эфир
Cryptocurrencies: 9529 / Markets: 107090
Market Cap: $ 3 881 274 157 427 / 24h Vol: $ 175 001 370 020 / BTC Dominance: 59.015510435686%

Н Новости

[Перевод] С новым годом: GPT в 500 строках на SQL

В минувшем году все только и говорили об ИИ и о том, как он может всё за вас сделать.

Мне нравится, когда кто-то или что-то работает за меня. Поэтому решил: попрошу-ка ChatGPT написать за меня новогодний пост:

"Эй, ChatGPT. А ты можешь реализовать большую языковую модель на SQL?"
"Нет, SQL не подходит для реализации больших языковых моделей. Язык SQL предназначен для выполнения запросов к данным, хранящимся в РСУБД и для управления этими запросами. Для реализации большой языковой модели требуются сложные алгоритмы, обширные датасеты, а зачастую – и фреймворки для глубокого обучения. SQL всеми этими возможностями не обладает."

Что ж, лишний раз убеждаешься, что, если хочешь что-то сделать хорошо – сделай это сам.

Давайте же воодушевимся этим оптимистическим планом и реализуем большую языковую модель на языке SQL.

Теория

Готовя этот пост, я вдохновлялся чудесной статьёй GPT in 60 Lines of NumPy, которую написал Джей Моди. В этой статье объяснено внутреннее устройство модели GPT, причём, это сделано гораздо лучше, чем могло бы получиться у меня. Поэтому для порядка вкратце напомню контекст.

Что представляет собой генеративная большая языковая модель с технической точки зрения?

Генеративная LLM – это функция. Она принимает на ввод текстовую последовательность символов (в терминологии ИИ она называется «промпт») и возвращает массив строк и чисел. Вот какова сигнатура этой функции:

llm(prompt: str) -> list[tuple[str, float]]

Это детерминированная функция. У неё под капотом выполняется серьёзная математика, но вся эта математика жёстко вшита в систему. Если вы будете многократно сообщать этой функции один и тот же ввод, то всякий раз будете получать от неё один и тот же вывод.

Тем, кому доводилось пользоваться ChatGPT и подобными продуктами, это может показаться удивительным, ведь система может давать разные ответы на один и тот же вопрос. Да, всё верно. Чуть ниже объясню, как это устроено.

Что за значения возвращает эта функция?

Нечто подобное:

	llm("I wish you a happy New")
 
0       (' Year', 0.967553)
1       (' Years', 0.018199688)
2       (' year', 0.003573329)
3       (' York', 0.003114716)
4       (' New', 0.0009022804)
…
50252   (' carbohyd', 2.3950911e-15)
50253   (' volunte', 2.2590102e-15)
50254   ('pmwiki', 1.369229e-15)
50255   (' proport', 1.1198108e-15)
50256   (' cumbers', 7.568147e-17)

Она возвращает массив кортежей. Каждый кортеж состоит из слова (скорее, строки) и числа. Число — это вероятность, с которой данное слово может оказаться промпте на следующей позиции. Модель «думает», что за фразой «I wish you a happy New…» («Поздравляю тебя с Новым…») с вероятностью 96,7% последует строка «Year» (…годом) – и так далее.

Слово «думает» выше поставлено в кавычках, так как, естественно, модель ни о чём не думает. Она механически возвращает массивы слов и чисел в соответствии с некоторой жёстко прописанной внутренней логикой.

Если эта система такая топорная и детерминированная, как же ей удаётся генерировать разные тексты?

Большие языковые модели используются в текстовых приложениях (чатботы, генераторы контента, кодинг-ассистенты, т.д.). Эти приложения раз за разом вызывают модель и выбирают подсказанное ею слово (с некоторой степенью произвольности). Последнее предложенное слово добавляется к промпту, и модель вызывается снова. Этот цикл продолжается до тех пор, пока не будет сгенерировано достаточное количество слов.

Накопленная таким образом последовательность слов выглядит как текст на естественном языке, с полноценной грамматикой, синтаксисом и даже мнимыми признаками интеллекта и логики. В данном отношении языковые модели подобны марковским цепям, работающим по такому же принципу.

На внутреннем уровне большая языковая модель устроена так, что каждое последующее предлагаемое слово естественным образом продолжало промпт, с учётом имеющейся грамматики, семантики и тональности. Оснастить функцию такой логикой стало возможно благодаря целой серии научных прорывов (и каторжной работе программистов). Удалось разработать целое семейство алгоритмов под названием GPT или «генеративные предобученные трансформеры».

Что такое "генеративный предобученный трансформер"

«Генеративный» означает, что алгоритм генерирует текст (рекурсивно достраивая продолжения к промпту как было показано выше).

«Трансформер» означает, что алгоритм использует конкретный тип нейронной сети. Сеть-трансформер была разработана компанией Google и описана в этой статье.

Термин «предобученный» требует небольшого исторического экскурса. Изначально считалось, что способность модели достраивать текст – это всего лишь предпосылка для получения более глубоких логических выводов в рамках специализированных задач (нужно улавливать логическую связь между фразами), классификации (например, угадать, сколько звёзд у отеля, проанализировав отзывы постояльцев), машинного перевода и т.д. Считалось, что две эти части алгоритма требуется обучать отдельно, и языковая часть позиционировалась как предобучение, подготовка к последующему обучению решать реальную задачу.

Как сформулировано в исходной статье по GPT:

Продемонстрировано, что возможно добиться больших успехов при решении таких задач путём генеративного предобучения языковой модели на разнородном корпусе неразмеченного текста с последующей тонкой настройкой различных параметров для различных слоёв сети (discriminative fine-tuning) в контексте каждой конкретной задачи.

Такие представления сохранялись до тех пор, пока не стало понятно, что для достаточно большой модели второй шаг зачастую излишен. Модель-трансформер, обученная исключительно генерации текстов, оказалась в состоянии без дополнительного обучения следовать человеческим инструкциям, содержащимся в этих текстах. Таким образом, не требовалось никакой дополнительной тонкой настройки.

Итак, с этими вопросами мы разобрались. Теперь давайте сосредоточимся на реализации.

Генерация

Вот что происходит, если мы пытаемся сгенерировать текст на основе промпта при помощи GPT2:

def generate(prompt: str) -> str:
  # Преобразует строку в список токенов.
  tokens = tokenize(prompt) # tokenize(prompt: str) -> list[int]
 
  while True:
 
    # Выполняет алгоритм.
    # Возвращает вероятностные значения для токенов: список из 50257 чисел с плавающей точкой, в сумме дающих 1.
    candidates = gpt2(tokens) # gpt2(tokens: list[int]) -> list[float]
 
    # Выбирает следующий токен из списка предложенных 
    next_token = select_next_token(candidates)
    # select_next_token(candidates: list[float]) -> int
 
    # Прикрепляет его к списку токенов
    tokens.append(next_token)
 
    # Решаем, хотим ли мы прекратить генерацию.
    # Здесь может быть счётчик токенов, задержка, стоп-слово или что-то ещё.
    if should_stop_generating():
      break
 
  # Преобразует список токенов в строку
  completion = detokenize(tokens) # detokenize(tokens: list[int]) -> str
  return completion

Давайте реализуем все эти элементы один за другим на языке SQL.

Токенизатор

Прежде, чем текст можно будет скормить нейронке, его требуется преобразовать в список чисел. Конечно, нас это не удивляет: именно такие задачи решаются при помощи текстовых кодировок вроде Unicode. Но обычный Unicode не слишком хорошо работает с нейронными сетями.

В сущности, нейронные сети построены на обширном перемножении матриц, и вся прогностическая сила нейронных сетей заключена в коэффициентах этих матриц. Некоторые из таких матриц содержат по строке на каждое из возможных значений в «алфавите», в других предусмотрено по строке на «символ».

В этом контексте смысл терминов «алфавит» и «символ» требуется пояснить. В Unicode длина «алфавита» составляет 149186 символов (именно столько отдельных точек Unicode существует на момент подготовки этой статьи), а в качестве «символа» может выступать, например: ﷽ (да, это одиночная точка Unicode, номер 65021, она соответствует целой фразе на арабском языке, особенно важной для мусульман). Отмечу, что эту фразу можно было бы записать и обычной арабской вязью. Таким образом, одному тексту может соответствовать несколько вариантов кодировки.

Давайте разберём эту ситуацию на примере слова "PostgreSQL". Если бы мы попытались закодировать её (преобразовать в массив чисел) при помощи Unicode, то у нас получилось бы 10 чисел, которые могли бы находиться в диапазоне от 1 до 149186. Таким образом, в нашей нейронной сети требовалось бы хранить матрицу из 149186 строк и совершить ряд вычислений над 10 строками из этой матрицы. Некоторые из этих строк (соответствующие буквам латиницы) использовались бы особенно часто, и в них было бы заключено много информации. Другие символы, например, смайлик какашки и таинственные символы из мёртвых языков едва ли вообще хоть раз использовались, но и на их хранение требуется место.

Естественно, мы хотим, чтобы оба эти числа, и длина «алфавита», и количество «символов» оставались как можно меньше. В идеале «символы» нашего алфавита должны быть распределены равномерно, а ещё мы по-прежнему хотим, чтобы наша кодировка не уступала по мощности Unicode.

Интуитивно понятно, что это можно сделать, присвоив уникальные номера тем последовательностям слов, которые часто встречаются в обрабатываемых нами текстах. В Unicode вышеупомянутая религиозная арабская фраза может быть закодирована либо как один кодовый символ, либо буква за буквой. Поскольку мы выкатываем нашу собственную кодировку, точно так же мы можем поступать с теми словами и фразами, которые важны в рамках нашей модели (например, они часто встречаются в текстах).

Например, можно было бы выделить конкретные числа для строк "Post", "greSQL" и "ing". Таким образом, оба слова "PostgreSQL" и "Posting" в нашем представлении будут иметь длину по 2. Конечно же, мы стремимся закреплять отдельные кодовые символы для более конкретных последовательностей и отдельных байт. Даже натыкаясь на тарабарщину или на иноязычный текст, такую информацию всё равно удастся закодировать, хотя, на это и потребуется больше времени.

В GPT2 используется вариант алгоритма под названием кодирование диадами (Byte Pair Encoding), предназначенный специально для этой цели. Токенизатор GPT2 использует словарь из 50257 кодовых точек (в терминологии ИИ — «токенов»), которые соответствуют различным последовательностям байт в UTF-8 (плюс отдельный токен, означающий «конец текста»).

Этот словарь мы собрали путём статистического анализа, выполненного таким образом:

1. Начинаем с простой кодировки из 256 токенов: по токену за байт.

2. Берём большой корпус текстов (предпочтительно именно тот, на котором будет обучаться модель).

3. Кодируем его.

4. Вычисляем, какая пара токенов встречается чаще всего. Пусть это будет 0x20 0x74 (пробел, за которым следует строчная "t").

5. Присваиваем следующее доступное значение (257) этой паре байт.

6. Повторяем шаги 3-5, теперь обращая внимание на последовательности байт. Если последовательность байт можно закодировать сложным токеном, то используем сложный токен. При явных неоднозначностях (например, "abc" может в одном случае быть закодировано как "a" + "bc" или как "ab" + "c"), то пользуйтесь той, номер которой меньше (это означает, что она была добавлена раньше и, следовательно, встречается чаще). Это нужно делать рекурсивно, пока все последовательности, которые можно сложить в один токен, будут сложены в один токен.

7. Выполним такое «складывание» 50000 раз подряд.

Число 50000 выбрано разработчиками более или менее произвольно. В других моделях количество токенов также держится в подобном диапазоне (от 30k до 100k).

На каждой итерации данного алгоритма в словарь будет добавляться новый токен, получаемый конкатенацией двух предыдущих. В конечном итоге получим 50256 токенов. Добавляем токен с фиксированным номером, означающий «конец текста» - и всё готово.

В версии алгоритма BTE для GPT2 есть ещё один уровень кодировки: в словаре токенов эти токены отображаются на строки, а не на массивы байт. Отображение с байт на строковые символы определяется в этой функции. Тот словарь, который она производит, находится в таблице encoder.

Рассмотрим, как реализовать токенизатор на SQL.

Данный токенизатор – неотъемлемая часть GPT2, и словарь токенов можно скачать с сайта OpenAI, равно как и оставшуюся часть модели. Нам понадобится импортировать её в таблицу tokenizer. В самом конце этого поста вы найдёте ссылку на репозиторий с кодом. Этот код автоматизирует заполнение таблиц базы данных, необходимых для работы с моделью.

В рамках рекурсивного подхода CTE разбиваем слово на токены (начиная с единичных байт) и объединяем наилучшие смежные пары, пока больше материала для объединения не останется. Само объединение происходит по принципу вложенного рекурсивного CTE.

Для примера я выбрал слово «Mississippilessly». Каждая запись в наборе результатов соответствует наилучшей найденной к данному моменту паре, которую можно «сложить», а также прогресс в обработке запроса.

WITH    RECURSIVE
        bpe AS
        (
        SELECT  (n + 1)::BIGINT AS position, character, TRUE AS continue, 1 AS step,
                NULL::INT AS token, NULL::TEXT AS combined
        FROM    CONVERT_TO('Mississippilessly', 'UTF-8') AS bytes
        CROSS JOIN LATERAL
                GENERATE_SERIES(0, LENGTH(bytes) - 1) AS n
        JOIN    encoder
        ON      byte = GET_BYTE(bytes, n)
        UNION ALL
        (
        WITH    RECURSIVE
                base AS
                (
                SELECT  *
                FROM    bpe
                WHERE   continue
                ),
                bn AS
                (
                SELECT  ROW_NUMBER() OVER (ORDER BY position) AS position,
                        continue,
                        character,
                        character || LEAD(character) OVER (ORDER BY position) AS cluster
                FROM    base
                ),
                top_rank AS
                (
                SELECT  tokenizer.*
                FROM    bn
                CROSS JOIN LATERAL
                        (
                        SELECT  *
                        FROM    tokenizer
                        WHERE   tokenizer.cluster = bn.cluster
                        LIMIT   1
                        ) tokenizer
                ORDER BY
                        token
                LIMIT   1
                ),
                breaks AS
                (
                SELECT  0::BIGINT AS position, 1 AS length
                UNION ALL
                SELECT  bn.position,
                        CASE WHEN token IS NULL THEN 1 ELSE 2 END
                FROM    breaks
                JOIN    bn
                ON      bn.position = breaks.position + length
                LEFT JOIN
                        top_rank
                USING   (cluster)
                )
        SELECT  position, character, token IS NOT NULL,
                (SELECT step + 1 FROM base LIMIT 1), token, top_rank.cluster
        FROM    breaks
        LEFT JOIN
                top_rank
        ON      1 = 1
        CROSS JOIN LATERAL
                (
                SELECT  STRING_AGG(character, '' ORDER BY position) AS character
                FROM    bn
                WHERE   bn.position >= breaks.position
                        AND bn.position < breaks.position + length
                ) bn
        WHERE   position > 0
        )
        )
SELECT  step, MAX(token) AS token, MAX(combined) AS combined, ARRAY_AGG(character ORDER BY position)
FROM    bpe
WHERE   continue
GROUP BY
        step
ORDER BY
        step
9afef708330d99752ee4da59fe3fceaf.png

На каждом шаге алгоритм BPE находит пару токенов, наиболее подходящую для слияния, и объединяет их (здесь в выводе вы видите такую слитую пару и ее ранг). При помощи такой процедуры размер пространства токенов можно сократить с 150k, присущих Unicode, до 50k, а количество токенов в данном конкретном слове с 17 до 5. В обоих случаях это отличное улучшение.

Когда приходится работать с множеством слов, токенизатор сначала разбивает текст на отдельные слова при помощи этого регулярного выражения и объединяет токены в каждом слове отдельно. К сожалению, при работе с PostgreSQL свойства символов Unicode не поддерживаются на уровне регулярных выражений, поэтому мне пришлось немного отладить систему (возможно, в процессе я угробил нормальную поддержку Unicode). Вот как результат выглядит в SQL:

WITH    input AS
        (
        SELECT  'PostgreSQL is great' AS prompt
        ),
        clusters AS
        (
        SELECT  part_position, bpe.*
        FROM    input
        CROSS JOIN LATERAL
                REGEXP_MATCHES(prompt, '''s|''t|''re|''ve|''m|''ll|''d| ?\w+| ?\d+| ?[^\s\w\d]+|\s+(?!\S)|\s+', 'g') WITH ORDINALITY AS rm (part, part_position)
        CROSS JOIN LATERAL
                (
                WITH    RECURSIVE
                        bpe AS
                        (
                        SELECT  (n + 1)::BIGINT AS position, character, TRUE AS continue
                        FROM    CONVERT_TO(part[1], 'UTF-8') AS bytes
                        CROSS JOIN LATERAL
                                GENERATE_SERIES(0, LENGTH(bytes) - 1) AS n
                        JOIN    encoder
                        ON      byte = GET_BYTE(bytes, n)
                        UNION ALL
                        (
                        WITH    RECURSIVE
                                base AS
                                (
                                SELECT  *
                                FROM    bpe
                                WHERE   continue
                                ),
                                bn AS
                                (
                                SELECT  ROW_NUMBER() OVER (ORDER BY position) AS position,
                                        continue,
                                        character,
                                        character || LEAD(character) OVER (ORDER BY position) AS cluster
                                FROM    base
                                ),
                                top_rank AS
                                (
                                SELECT  tokenizer.*
                                FROM    bn
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  *
                                        FROM    tokenizer
                                        WHERE   tokenizer.cluster = bn.cluster
                                        LIMIT   1
                                        ) tokenizer
                                ORDER BY
                                        token
                                LIMIT   1
                                ),
                                breaks AS
                                (
                                SELECT  0::BIGINT AS position, 1 AS length
                                UNION ALL
                                SELECT  bn.position,
                                        CASE WHEN token IS NULL THEN 1 ELSE 2 END
                                FROM    breaks
                                JOIN    bn
                                ON      bn.position = breaks.position + length
                                LEFT JOIN
                                        top_rank
                                USING   (cluster)
                                )
                        SELECT  position, character, token IS NOT NULL
                        FROM    breaks
                        LEFT JOIN
                                top_rank
                        ON      1 = 1
                        CROSS JOIN LATERAL
                                (
                                SELECT  STRING_AGG(character, '' ORDER BY position) AS character
                                FROM    bn
                                WHERE   bn.position >= breaks.position
                                        AND bn.position < breaks.position + length
                                ) bn
                        WHERE   position > 0
                        )
                        )
                SELECT  position, character AS cluster
                FROM    bpe
                WHERE   NOT continue
                ) bpe
        ),
        tokens AS
        (
        SELECT  token, cluster
        FROM    clusters
        JOIN    tokenizer
        USING   (cluster)
        )
SELECT  *
FROM    tokens
05c9a2cc7365cf2c4798ffb6c884c433.png

Странный символ Ġ соответствует пробелу.

Этот запрос токенизирует промпт и преобразует его в массив чисел. Вот промпт и готов к долгому пути сквозь слои модели.

Векторные представления

Токены - это представления единиц естественного языка (в целом, примерно 0,75 слова на токен), поэтому любая модель, претендующая на успешное автозавершение текстов, должна каким-то образом кодировать отношения между этими единицами. Даже в изоляции части речи обладают наборами ортогональных свойств.

Возьмём, к примеру, слово «subpoena» (повестка) – оказывается, в токенизаторе GPT2 на него отведён целый токен. С точки зрения английского языка – это существительное? Да, бесспорно. А глагол? Да, и глаголом может быть. А прилагательным? Уже не очень, но, в принципе, контекст придумать можно. Это канцелярит? Чёрт возьми, ещё какой. И так далее.

Все эти свойства ортогональны, то есть, не зависят друг от друга. Слово может быть официозным существительным, но не глаголом и не прилагательным. В английском возможны любые сочетания этих признаков.

Сущности с ортогональными свойствами лучше всего кодируются в виде векторных представлений. Пусть у нас будет не единственное свойство (например, номер), а много. При желании мы можем крутить ими как захотим. Например, фраза «В судебном решении, которое процитировал адвокат, фигурирует…» с высокой вероятностью продолжается каким-то существительным, которое, не менее вероятно, стилистически относится к канцеляриту. Нас совершенно не интересует, есть ли у этого слова маргинальная семантика «глагол», либо «прилагательное», либо «цветок».

В математике отображение сравнительно узких значений на более широкие пространства (например, ID токенов на векторы) называется векторным представлением. Именно этим мы здесь и занимаемся.

Как решить, какие именно свойства окажутся представлены в этих векторах? А мы не будем. Мы просто предоставим достаточно векторного пространства для каждого токена и понадеемся, что модель на этапе обучения получит достаточно информации, чтобы заполнить все эти измерения осмысленным содержимым. В GPT2 под векторы отводится 768 измерений. Заранее (и, на самом деле, даже в ретроспективе) не известно, какое свойство слова будет закодировано, например, в измерении 247. Определённо, что-то закодировано там будет, но что именно – сказать сложно.

Какие свойства каждого из токенов мы внедрим в векторное пространство? Любые, которые помогут нам спрогнозировать, каков будет следующий токен.

ID токена? Естественно. Разные токены означают разные вещи.

Позиция токена в тексте? Да, пожалуйста. «Сине-фиолетовый» и «фиолетово-синий» — не одно и то же.

Взаимные отношения токенов? Естественно! Это, пожалуй, наиважнейшая часть задачи, и блок внимания (Attention) в архитектуре трансформера как раз впервые позволил решить эту часть задачи правильно.

Токены и позиции легко поддаются векторному представлению. Допустим, есть фраза «PostgreSQL is great», о которой заранее известно, что она отображается на четыре токена: [6307, 47701, 318, 1049].

Среди прочих параметров GPT2 есть две матрицы, называющиеся WTE (векторное представление лексических токенов) и WPE (векторное представление позиций слов). Как понятно из названия, в первом хранятся векторные представления токенов, а во втором – векторные представления их позиций. Конкретные значения, которыми окажутся заполнены эти представления, набираются в результате обучения модели, в данном случае GPT2. Насколько известно, в таблицах баз данных wte и wpe есть константы. Размер

WTE равен 50257×768, а размер WPE равен 1024×768. Второе означает, что в GPT2 можно использовать промпт, состоящий не более чем из 1024 токенов. Если записать в промпт больше токенов, мы просто не сможем вытянуть для токенов сверх 1024 их позиционные представления. Это архитектурный аспект (в терминологии ИИ – «гиперпараметр») модели, устанавливаемый на этапе её проектирования, и обучением его изменить невозможно. Когда говорят о «контекстном окне» большой языковой модели, имеют в виду именно это число.

Токен 6307 стоит у нас на месте 0, токен 47701 на месте 1, токен 318 на месте 2, а токен 1049 на месте 3. Для каждого из этих токенов и их позиций у нас по два вектора: один из WTE и один из WPE. Их требуется сложить. В результате получатся четыре вектора, которые послужат вводом для следующей части алгоритма: так работает нейронная сеть прямого распространения, в которой действует механизм внимания.

Для работы с SQL мы воспользуемся pgvector, это расширение PostgreSQL.

Небольшая оговорка: код для этого поста я пишу на обычном SQL, иногда вставляю чистые SQL-функции в качестве вспомогательных элементов. В рамках этого поста осуществить такое не составит труда, равно как определить векторные операции над массивами. Это делается за счёт небольшого снижения производительности (и уже было реализовано в версии 1 и работало, пусть и медленно). С распространением ИИ и ростом влияния векторных баз данных pgvector или эквивалентный механизм определённо укоренится в ядре PostgreSQL в течение двух-трёх релизов. Я просто решил оседлать волну, которая принесёт меня в будущее.

Вот как это делается в SQL:

WITH    embeddings AS
        (
        SELECT  place, values
        FROM    UNNEST(ARRAY[6307, 47701, 318, 1049]) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        )
SELECT  place, (values::REAL[])[0:5]
FROM    embeddings
2eb33d41dc02c972da3f448c339b4366.png

(Чтобы вывод был короче, в этом запросе показаны только первые 5 измерений для каждого вектора).

Внимание

Та часть трансформерной архитектуры, благодаря которой она действительно феерит, называется «механизм внутреннего внимания» (self-attention). Он был впервые описан в 2017 году в статье «Attention is all you need» под авторством Васмани и др. Пожалуй, это наиизвестнейшая статья об искусственном интеллекте, и с тех пор её название превратилось в болванку (клише для именования других статей).

Итак, вероятно, к настоящему моменту у нас уже есть несколько векторов, в которых закодированы определённые синтаксические и семантические свойства слов, содержащихся в нашем промпте. Нам каким-то образом нужно донести эти свойства до последнего вектора. Сейчас будет небольшой спойлер: именно в последнем векторе будет храниться представление того слова, которым продолжится фраза.

Во фразе вида a «I looked at the violet and saw that it was not the usual …» (я взглянул на лиловое пятно и заметил, что это не обычный…), на месте многоточия должно быть что-то, увиденное вами (об этом свидетельствует слово «saw»), обладающее свойством «лиловости» и при этом «необычное» (сочетаем токены «не» и «обычный», тем самым меняя знак измерения «обычность» на противоположный). Можно привести такую реалистичную аналогию: человек читает книгу на иностранном языке, о котором имеет некоторое представление, но свободно им не владеет. В таком случае человеку придётся аккуратно сплетать каждое слово со следующим, и, если он не обратит внимания на ключевую часть фразы, то всю фразу поймёт неправильно.

Чтобы обеспечить такой перенос значения от одного токена к другому, требуется разрешить векторам всех токенов влиять друг на друга. Так, возвращаясь к английскому: если мы хотим наполнить слово «it» конкретной семантикой, то какая часть семантики должна поступить от предыдущих векторов в промпте, а какая – остаться от самого слова «it»?

В данной модели эта задача решается при помощи 12 наборов матриц, именуемых Q (запрос), K (ключ) и V (значение). В каждой из них по 64 столбца. Их мы получаем из векторных представлений через 768×2304-мерное линейное преобразование c_attn, веса и смещения которого хранятся в таблицах c_attn_w и c_attn_b.

Результатом c_attn является матрица со n_token строк и 2304 столбцами (3×12×64). Она состоит из 12 Q-матриц, 12 K-матриц и 12 V-матриц, упорядоченных горизонтально именно в таком порядке.

Каждый набор, состоящий из Q, K и V, называется «головой». С их помощью выполняется шаг под названием «многоголовое условное внутреннее внимание», на котором вычисляется функция внимания.

Она вычисляется по этой формуле:

b7a3f0ee8816d3c95951491a32765034.png

где softmax – это функция нормализации весов. Она определяется так:

f63cb8d1f794504cc4a9512b4ba3ef4b.png

M - это матрица-константа, именуемая «условной маской». Она определяется так:

6663f6f6cf9689a78b9b50a827fe4b53.png

Функция Softmax обнуляет все отрицательные бесконечности.

Почему необходимо использовать маски?

В промпте из предыдущего примера было 4 токена, и модель первым делом вычисляла 4 векторных представления для 4 этих токенов. По мере дальнейшей работы модели над этими векторами будет выполнено много вычислений, но в основном они пойдут независимо и параллельно. Изменения в одном векторе не затронут другие векторы, как если бы иных векторов не существовало. Векторы влияют друг на друга только в пределах блока внутреннего внимания.

Как только модель справится с математикой, претенденты на роль следующего токена будут выбираться исключительно исходя из последнего векторного представления. Вся информация должна направляться именно к этому последнему вектору, а не прочь от него. Временные значения последнего вектора не должны влиять на временные значения предыдущих векторов, использовавшихся в ходе прямого распространения по данной модели.

Именно поэтому мы «маскируем» позднейшие векторы, чтобы они не влияли на более ранние через этот конкретный канал. В этом и заключается условность «многоголового условного внутреннего внимания».

Почему матрицы называются "запрос", "ключ" и "значение"?

Честно говоря, даже не уверен, что это хорошая аналогия. Но, тем не менее, попытаюсь объяснить, какова логика таких названий.

Как правило, в машинном обучении при вычислениях не должны использоваться циклы переменной длины или ветвление инструкций. Все задачи должны решаться компоновкой простых аналитических функций (сложение, умножение, возведение в степень, логарифмы и тригонометрия). Так обеспечивается обратное распространение для эффективной реализации которого требуются такие технологии, как автоматическое дифференцирование.

Математическая модель хранилища ключей и значений заключена в таком выражении:

de4db9304ce9fe7850fa07f4aa82a4db.png

Но это не гладкая дифференцируемая функция, и с обратным распространением она работать не будет. Чтобы формула работала, необходимо получить гладкую функцию, которая была бы близка к v, когда k близка к q, и близка к 0 в иных случаях.

Для этой цели отлично подходит Гауссово распределение («колоколообразная кривая») "bell curve"), отмасштабированная к v с ожиданием k и достаточно небольшим стандартным отклонением:

cb7ce72dab8bcafebcd14cf861b908a2.png

где σ - произвольный параметр, определяющий, насколько «остра» колоколообразная кривая.

Если взять в векторном пространстве с достаточно большим количеством измерений фиксированный вектор K и несколько векторов Q, которые случайным и равномерным образом отклоняются от K в каждом из измерений, то их скалярные произведения сложатся именно в колоколообразную кривую. Соответственно, в векторном пространстве «дифференцируемое хранилище ключей и значений» можно смоделировать при помощи выражения

673f867f6aacaf8ba018b96d5399f9e2.png

того самого, которым мы пользовались в нашей функции внимания.

Опять же, эта аналогия немного натянута. Лучше не слишком зацикливаться на концептуальной составляющей внимания, то есть, на потоке операций, хеш-таблицах и пр. Можете считать, что они просто вдохновили меня на описанный математический фокус, который я догадался протестировать – и на практике он оказался весьма хорош.

Проиллюстрирую этот шаг:

WITH    embeddings AS
        (
        SELECT  place, values
        FROM    UNNEST(ARRAY[6307, 47701, 318, 1049]) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        ),
        c_attn_w AS
        (
        SELECT  *
        FROM    c_attn_w
        WHERE   block = 0
        ),
        c_attn_b AS
        (
        SELECT  *
        FROM    c_attn_b
        WHERE   block = 0
        ),
        ln_1_g AS
        (
        SELECT  *
        FROM    ln_1_g
        WHERE   block = 0
        ),
        ln_1_b AS
        (
        SELECT  *
        FROM    ln_1_b
        WHERE   block = 0
        ),
        mha_norm AS
        (
        SELECT  place, mm.values + c_attn_b.values AS values
        FROM    (
                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                FROM    (
                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                        FROM    (
                                SELECT  place, norm.values
                                FROM    embeddings
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  AVG(value) AS mean,
                                                VAR_POP(value) AS variance
                                        FROM    UNNEST(values::REAL[]) value
                                        ) agg
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                        ) norm
                                ) agg
                        CROSS JOIN
                                ln_1_b
                        CROSS JOIN
                                ln_1_g
                        ) layer_norm
                CROSS JOIN
                        c_attn_w
                GROUP BY
                        place
                ) mm
        CROSS JOIN
                c_attn_b
        ),
        head AS
        (
        SELECT  place,
                (values::REAL[])[1:64]::VECTOR(64) AS q,
                (values::REAL[])[1 + 768:64 + 768]::VECTOR(64) AS k,
                (values::REAL[])[1 + 1536:64 + 1536]::VECTOR(64) AS v
        FROM    mha_norm
        ),
        sm_input AS
        (
        SELECT  h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
        FROM    head h1
        CROSS JOIN
                head h2
        ),
        sm_diff AS
        (
        SELECT  x, y, value - MAX(value) OVER (PARTITION BY x) AS diff
        FROM    sm_input
        ),
        sm_exp AS
        (
        SELECT  x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
        FROM    sm_diff
        ),
        softmax AS
        (
        SELECT  x, y AS place, e / SUM(e) OVER (PARTITION BY x) AS value
        FROM    sm_exp
        ),
        attention AS
        (
        SELECT  place, (ARRAY_AGG(value ORDER BY ordinality))[:3] AS values
        FROM    (
                SELECT  x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * head.v) AS values
                FROM    softmax
                JOIN    head
                USING   (place)
                GROUP BY
                        x
                ) q
        CROSS JOIN LATERAL
                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
        GROUP BY
                place
        )
SELECT  place,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((q::REAL[])[:3]) AS n) AS q,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((k::REAL[])[:3]) AS n) AS k,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((v::REAL[])[:3]) AS n) AS v,
        matrix,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((values::REAL[])[:3]) AS n) AS attention
FROM    head
JOIN    attention
USING   (place)
JOIN    (
        SELECT  x AS place, STRING_AGG(CASE WHEN value > 0 THEN TO_CHAR(value, '0.00') ELSE '    0' END, ' ' ORDER BY place) AS matrix
        FROM    softmax
        GROUP BY
                x
        ) softmax_grouped
USING   (place)
6f968ec80257c3b21a63e633a5c9d13f.png

Вот что мы сделали:

1. Прежде, чем вычислять функцию внимания, мы нормализовали векторы при помощи линейного преобразования R’ = RГ1 + B1. Матрица Г1 и вектор B1называются, соответственно, «масштабирование» и «сдвиг». Это параметры, изученные моделью, которые хранятся в таблицах ln_1_g и ln_1_b

2. Мы показываем только первую голову первого слоя, используемого в этом алгоритме. После того, как умножим векторы на изученные коэффициенты из c_attn_w и c_attn_b («вес» и «смещение»), порежем полученные в результате 2304-мерные векторы. Возьмём 64-мерные векторы, начинающиеся в позициях 0, 768 и 1536. Они соответствуют векторам Q, K и V первой головы.

3. EXP в PostgreSQL плохо работает с очень малыми числами, и именно поэтому мы прибегаем к нулю, если EXP получает аргумент менее -745,13.

4. Для каждого вектора мы показываем только первые три элемента. Марицу внимания показываем полностью.

Итак, значение первого вектора копируется в вывод как есть (так мы поступаем с первым значением на каждом слое алгоритма). Это означает, что, когда модель уже обучена, результирующее векторное представление первого токена будет определяться только значением первого токена. В принципе, на этапе рекурсивного вывода, пока токены только добавляются в промпт, по сравнению с предыдущей итерацией изменится лишь самое последнее векторное представление в выводе. Именно поэтому применяется условная маска.

Заглянем немного вперёд: блок внимания – это единственное место во всём алгоритме, где токены могут влиять друг на друга на этапе прямого прохода. Поскольку на данном этапе мы не позволяем позднейшим токенам влиять на более раннее (и отключили такую возможность), все вычисления, проделанные над более ранними токенами, можно повторно использовать на разных итерациях прямого прохода по модели.

Напомню, модель при работе добавляет токены в промпт. Если исходный (токенизированный) промпт – это «Post greSQL Ġis Ġgreat», а следующий, например, будет «Post greSQL Ġis Ġgreat Ġfor», то все результаты вычислений, проделанных над первыми четырьмя токенами, можно будет повторно использовать и в новом промпте. Они никогда не изменятся, независимо от того, что ещё будет прикреплено к этому промпту.

В статье Джея Муди, которую я привёл для примера, этот аспект не задействуется (равно как и в моей статье – для простоты), но в оригинальной реализации GPT2 он задействован.

Закончив работу со всеми головами, получаем 12 матриц. Каждая из них будет по 64 столбца в ширину и по n_tokens строк в высоту. Чтобы вновь отобразить её на размерность векторных представлений (768), нам просто нужно составить эти матрицы горизонтально.

Последний этап обработки многоголового внимания связан с проецированием значений путём линейного преобразования в том же самом измерении. Соответствующие веса и смещения хранятся в таблицах c_proj_w и c_proj_b.

Ниже приведён полный код, описывающий реализацию многоголового внимания в первом слое:

WITH    embeddings AS
        (
        SELECT  place, values
        FROM    UNNEST(ARRAY[6307, 47701, 318, 1049]) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        ),
        c_proj_w AS
        (
        SELECT  *
        FROM    c_proj_w
        WHERE   block = 0
        ),
        c_proj_b AS
        (
        SELECT  *
        FROM    c_proj_b
        WHERE   block = 0
        ),
        mlp_c_fc_w AS
        (
        SELECT  *
        FROM    mlp_c_fc_w
        WHERE   block = 0
        ),
        mlp_c_fc_b AS
        (
        SELECT  *
        FROM    mlp_c_fc_b
        WHERE   block = 0
        ),
        mlp_c_proj_w AS
        (
        SELECT  *
        FROM    mlp_c_proj_w
        WHERE   block = 0
        ),
        mlp_c_proj_b AS
        (
        SELECT  *
        FROM    mlp_c_proj_b
        WHERE   block = 0
        ),
        c_attn_w AS
        (
        SELECT  *
        FROM    c_attn_w
        WHERE   block = 0
        ),
        c_attn_b AS
        (
        SELECT  *
        FROM    c_attn_b
        WHERE   block = 0
        ),
        ln_1_g AS
        (
        SELECT  *
        FROM    ln_1_g
        WHERE   block = 0
        ),
        ln_1_b AS
        (
        SELECT  *
        FROM    ln_1_b
        WHERE   block = 0
        ),
        mha_norm AS
        (
        SELECT  place, mm.values + c_attn_b.values AS values
        FROM    (
                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                FROM    (
                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                        FROM    (
                                SELECT  place, norm.values
                                FROM    embeddings
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  AVG(value) AS mean,
                                                VAR_POP(value) AS variance
                                        FROM    UNNEST(values::REAL[]) value
                                        ) agg
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                        ) norm
                                ) agg
                        CROSS JOIN
                                ln_1_b
                        CROSS JOIN
                                ln_1_g
                        ) layer_norm
                CROSS JOIN
                        c_attn_w
                GROUP BY
                        place
                ) mm
        CROSS JOIN
                c_attn_b
        ),
        heads AS
        (
        SELECT  place, head,
                (values::REAL[])[(head * 64 + 1):(head * 64 + 64)]::VECTOR(64) AS q,
                (values::REAL[])[(head * 64 + 1 + 768):(head * 64 + 64 + 768)]::VECTOR(64) AS k,
                (values::REAL[])[(head * 64 + 1 + 1536):(head * 64 + 64 + 1536)]::VECTOR(64) AS v
        FROM    mha_norm
        CROSS JOIN
                GENERATE_SERIES(0, 11) head
        ),
        sm_input AS
        (
        SELECT  head, h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
        FROM    heads h1
        JOIN    heads h2
        USING   (head)
        ),
        sm_diff AS
        (
        SELECT  head, x, y, value - MAX(value) OVER (PARTITION BY head, x) AS diff
        FROM    sm_input
        ),
        sm_exp AS
        (
        SELECT  head, x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
        FROM    sm_diff
        ),
        softmax AS
        (
        SELECT  head, x, y AS place, e / SUM(e) OVER (PARTITION BY head, x) AS value
        FROM    sm_exp
        ),
        attention AS
        (
        SELECT  place, ARRAY_AGG(value ORDER BY head * 64 + ordinality)::VECTOR(768) AS values
        FROM    (
                SELECT  head, x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * heads.v) AS values
                FROM    softmax
                JOIN    heads
                USING   (head, place)
                GROUP BY
                        head, x
                ) q
        CROSS JOIN LATERAL
                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
        GROUP BY
                place
        ),
        mha AS
        (
        SELECT  place, w.values + c_proj_b.values AS values
        FROM    (
                SELECT  attention.place, ARRAY_AGG(INNER_PRODUCT(attention.values, c_proj_w.values) ORDER BY c_proj_w.place)::VECTOR(768) AS values
                FROM    attention
                CROSS JOIN
                        c_proj_w
                GROUP BY
                        attention.place
                ) w
        CROSS JOIN
                c_proj_b
        )
SELECT  place,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((values::REAL[])[:10]) AS n) AS q
FROM    mha
d5d6733a783e738538d2d95ec2114fca.png

Прежде чем результаты многоголового внимания будут переданы на следующий шаг, к ним нужно добавить оригинальные входные данные. Этот фокус был описан ещё в исходной статье по трансформерам. Предполагается, что он должен помочь с затухающими и взрывными градиентами.

Это распространённая проблема при обучении: иногда градиенты параметров оказываются слишком малы или слишком велики. Если менять их при очередной итерации обучения, это также очень слабо влияет на функцию потерь (поэтому модель сходится очень медленно). Бывает и обратный эффект: даже минимальное изменение отбрасывает функцию потерь слишком далеко от локального минимума, тем самым обесценивая всю работу по обучению.

Прямое распространение

Именно в этом и заключается работа глубоких нейронных сетей. Большая часть параметров модели на самом деле используется именно на этом этапе.

Здесь мы имеем дело с многослойным перцептроном, у которого три слоя (768, 3072, 768), а в качестве функции активации используем линейную единицу гауссовской ошибки (GELU):

1582eb4081148863170cf26deff796b0.png

Как показывает практика, эта функция даёт очень хорошие результаты при работе с глубокими нейронными сетями. Её можно аналитически аппроксимировать вот так:

4e7d6916854cc78e19fab9621165d0de.png

Изученные параметры линейного преобразования для связей в слое называются c_fc (768 → 3072) и c_proj (3072 → 768). Значения с первого слоя сначала нормализуются при помощи коэффициентов, содержащихся в изученном параметре ln_2. По завершении этапа прямого распространения его ввод вновь добавляется к выводу. Этот элемент также входит в самый первый вариант дизайна трансформеров. Весь этап прямого распространения выглядит так:

А вот как это делается на SQL:

WITH    embeddings AS
        (
        SELECT  place, values
        FROM    UNNEST(ARRAY[6307, 47701, 318, 1049]) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        ),
        c_proj_w AS
        (
        SELECT  *
        FROM    c_proj_w
        WHERE   block = 0
        ),
        c_proj_b AS
        (
        SELECT  *
        FROM    c_proj_b
        WHERE   block = 0
        ),
        mlp_c_fc_w AS
        (
        SELECT  *
        FROM    mlp_c_fc_w
        WHERE   block = 0
        ),
        mlp_c_fc_b AS
        (
        SELECT  *
        FROM    mlp_c_fc_b
        WHERE   block = 0
        ),
        mlp_c_proj_w AS
        (
        SELECT  *
        FROM    mlp_c_proj_w
        WHERE   block = 0
        ),
        mlp_c_proj_b AS
        (
        SELECT  *
        FROM    mlp_c_proj_b
        WHERE   block = 0
        ),
        c_attn_w AS
        (
        SELECT  *
        FROM    c_attn_w
        WHERE   block = 0
        ),
        c_attn_b AS
        (
        SELECT  *
        FROM    c_attn_b
        WHERE   block = 0
        ),
        ln_1_g AS
        (
        SELECT  *
        FROM    ln_1_g
        WHERE   block = 0
        ),
        ln_1_b AS
        (
        SELECT  *
        FROM    ln_1_b
        WHERE   block = 0
        ),
        ln_2_b AS
        (
        SELECT  *
        FROM    ln_2_b
        WHERE   block = 0
        ),
        ln_2_g AS
        (
        SELECT  *
        FROM    ln_2_g
        WHERE   block = 0
        ),
        mha_norm AS
        (
        SELECT  place, mm.values + c_attn_b.values AS values
        FROM    (
                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                FROM    (
                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                        FROM    (
                                SELECT  place, norm.values
                                FROM    embeddings
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  AVG(value) AS mean,
                                                VAR_POP(value) AS variance
                                        FROM    UNNEST(values::REAL[]) value
                                        ) agg
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                        ) norm
                                ) agg
                        CROSS JOIN
                                ln_1_b
                        CROSS JOIN
                                ln_1_g
                        ) layer_norm
                CROSS JOIN
                        c_attn_w
                GROUP BY
                        place
                ) mm
        CROSS JOIN
                c_attn_b
        ),
        heads AS
        (
        SELECT  place, head,
                (values::REAL[])[(head * 64 + 1):(head * 64 + 64)]::VECTOR(64) AS q,
                (values::REAL[])[(head * 64 + 1 + 768):(head * 64 + 64 + 768)]::VECTOR(64) AS k,
                (values::REAL[])[(head * 64 + 1 + 1536):(head * 64 + 64 + 1536)]::VECTOR(64) AS v
        FROM    mha_norm
        CROSS JOIN
                GENERATE_SERIES(0, 11) head
        ),
        sm_input AS
        (
        SELECT  head, h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
        FROM    heads h1
        JOIN    heads h2
        USING   (head)
        ),
        sm_diff AS
        (
        SELECT  head, x, y, value - MAX(value) OVER (PARTITION BY head, x) AS diff
        FROM    sm_input
        ),
        sm_exp AS
        (
        SELECT  head, x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
        FROM    sm_diff
        ),
        softmax AS
        (
        SELECT  head, x, y AS place, e / SUM(e) OVER (PARTITION BY head, x) AS value
        FROM    sm_exp
        ),
        attention AS
        (
        SELECT  place, ARRAY_AGG(value ORDER BY head * 64 + ordinality)::VECTOR(768) AS values
        FROM    (
                SELECT  head, x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * heads.v) AS values
                FROM    softmax
                JOIN    heads
                USING   (head, place)
                GROUP BY
                        head, x
                ) q
        CROSS JOIN LATERAL
                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
        GROUP BY
                place
        ),
        mha AS
        (
        SELECT  place, w.values + c_proj_b.values + embeddings.values AS values
        FROM    (
                SELECT  attention.place, ARRAY_AGG(INNER_PRODUCT(attention.values, c_proj_w.values) ORDER BY c_proj_w.place)::VECTOR(768) AS values
                FROM    attention
                CROSS JOIN
                        c_proj_w
                GROUP BY
                        attention.place
                ) w
        CROSS JOIN
                c_proj_b
        JOIN    embeddings
        USING   (place)
        ),
        ffn_norm AS
        (
        SELECT  place, agg.values * ln_2_g.values + ln_2_b.values AS values
        FROM    (
                SELECT  place, norm.values
                FROM    mha
                CROSS JOIN LATERAL
                        (
                        SELECT  AVG(value) AS mean,
                                VAR_POP(value) AS variance
                        FROM    UNNEST(values::REAL[]) value
                        ) agg
                CROSS JOIN LATERAL
                        (
                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                        ) norm
                ) agg
        CROSS JOIN
                ln_2_b
        CROSS JOIN
                ln_2_g
        ),
        ffn_a AS
        (
        SELECT  gelu.place, gelu.values
        FROM    (
                SELECT  place, w.values + mlp_c_fc_b.values AS values
                FROM    (
                        SELECT  ffn_norm.place, ARRAY_AGG(INNER_PRODUCT(ffn_norm.values, mlp_c_fc_w.values) ORDER BY mlp_c_fc_w.place)::VECTOR(3072) AS values
                        FROM    ffn_norm
                        CROSS JOIN
                                mlp_c_fc_w
                        GROUP BY
                                ffn_norm.place
                        ) w
                CROSS JOIN
                        mlp_c_fc_b
                ) v
        CROSS JOIN LATERAL
                (
                SELECT  place, ARRAY_AGG(0.5 * value * (1 + TANH(0.797884560802 * (value + 0.044715 * value*value*value))) ORDER BY ordinality)::VECTOR(3072) AS values
                FROM    UNNEST(values::REAL[]) WITH ORDINALITY n (value, ordinality)
                GROUP BY
                        place
                ) gelu
        ),
        ffn AS
        (
        SELECT  place, w.values + mlp_c_proj_b.values + mha.values AS values
        FROM    (
                SELECT  ffn_a.place, ARRAY_AGG(INNER_PRODUCT(ffn_a.values, mlp_c_proj_w.values) ORDER BY mlp_c_proj_w.place)::VECTOR(768) AS values
                FROM    ffn_a
                CROSS JOIN
                        mlp_c_proj_w
                GROUP BY
                        ffn_a.place
                ) w
        CROSS JOIN
                mlp_c_proj_b
        JOIN    mha
        USING   (place)
        )
SELECT  place,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((values::REAL[])[:10]) AS n) AS q
FROM    ffn
9080f54c68ca45f66b84cda8eba312be.png

Именно этот вывод получаем из первого блока GPT2.

Блоки

Операции, рассмотренные на предыдущих этапах, повторяются в каждом из слоёв (именуемых «блоками»). Блоки выстраиваются в виде конвейера, так, что вывод из предыдущего блока сразу подаётся на ввод следующему. У каждого из блоков – свой набор изученных параметров.

В SQL потребуется соединить блоки при помощи рекурсивного CTE.

Как только финальный блок выдаст значения, результат потребуется нормализовать при помощи изученного параметра ln_f.

Вот как в итоге будет выглядеть модель:

7699aba465e680cfde43e9ff6ec6a2b2.png

А вот как результат выглядит на SQL:

WITH    RECURSIVE
        initial AS
        (
        SELECT  ARRAY[6307, 47701, 318, 1049] AS input
        ),
        hparams AS
        (
        SELECT  12 AS n_block
        ),
        embeddings AS
        (
        SELECT  place, values
        FROM    initial
        CROSS JOIN
                hparams
        CROSS JOIN LATERAL
                UNNEST(input) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        ),
        transform AS
        (
        SELECT  0 AS block, place, values
        FROM    embeddings
        UNION ALL
        (
        WITH    previous AS
                (
                SELECT  *
                FROM    transform
                )
        SELECT  block + 1 AS block, transformed_layer.*
        FROM    hparams
        CROSS JOIN LATERAL
                (
                SELECT  block
                FROM    previous
                WHERE   block < 12
                LIMIT   1
                ) q
        CROSS JOIN LATERAL
                (
                WITH    ln_2_b AS
                        (
                        SELECT  *
                        FROM    ln_2_b
                        WHERE   block = q.block
                        ),
                        ln_2_g AS
                        (
                        SELECT  *
                        FROM    ln_2_g
                        WHERE   block = q.block
                        ),
                        c_proj_w AS
                        (
                        SELECT  *
                        FROM    c_proj_w
                        WHERE   block = q.block
                        ),
                        c_proj_b AS
                        (
                        SELECT  *
                        FROM    c_proj_b
                        WHERE   block = q.block
                        ),
                        mlp_c_fc_w AS
                        (
                        SELECT  *
                        FROM    mlp_c_fc_w
                        WHERE   block = q.block
                        ),
                        mlp_c_fc_b AS
                        (
                        SELECT  *
                        FROM    mlp_c_fc_b
                        WHERE   block = q.block
                        ),
                        mlp_c_proj_w AS
                        (
                        SELECT  *
                        FROM    mlp_c_proj_w
                        WHERE   block = q.block
                        ),
                        mlp_c_proj_b AS
                        (
                        SELECT  *
                        FROM    mlp_c_proj_b
                        WHERE   block = q.block
                        ),
                        c_attn_w AS
                        (
                        SELECT  *
                        FROM    c_attn_w
                        WHERE   block = q.block
                        ),
                        c_attn_b AS
                        (
                        SELECT  *
                        FROM    c_attn_b
                        WHERE   block = q.block
                        ),
                        ln_1_g AS
                        (
                        SELECT  *
                        FROM    ln_1_g
                        WHERE   block = q.block
                        ),
                        ln_1_b AS
                        (
                        SELECT  *
                        FROM    ln_1_b
                        WHERE   block = q.block
                        ),
                        mha_norm AS
                        (
                        SELECT  place, mm.values + c_attn_b.values AS values
                        FROM    (
                                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                                FROM    (
                                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                                        FROM    (
                                                SELECT  place, norm.values
                                                FROM    previous
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  AVG(value) AS mean,
                                                                VAR_POP(value) AS variance
                                                        FROM    UNNEST(values::REAL[]) value
                                                        ) agg
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                                        ) norm
                                                ) agg
                                        CROSS JOIN
                                                ln_1_b
                                        CROSS JOIN
                                                ln_1_g
                                        ) layer_norm
                                CROSS JOIN
                                        c_attn_w
                                GROUP BY
                                        place
                                ) mm
                        CROSS JOIN
                                c_attn_b
                        ),
                        heads AS
                        (
                        SELECT  place, head,
                                (values::REAL[])[(head * 64 + 1):(head * 64 + 64)]::VECTOR(64) AS q,
                                (values::REAL[])[(head * 64 + 1 + 768):(head * 64 + 64 + 768)]::VECTOR(64) AS k,
                                (values::REAL[])[(head * 64 + 1 + 1536):(head * 64 + 64 + 1536)]::VECTOR(64) AS v
                        FROM    mha_norm
                        CROSS JOIN
                                GENERATE_SERIES(0, 11) head
                        ),
                        sm_input AS
                        (
                        SELECT  head, h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
                        FROM    heads h1
                        JOIN    heads h2
                        USING   (head)
                        ),
                        sm_diff AS
                        (
                        SELECT  head, x, y, value - MAX(value) OVER (PARTITION BY head, x) AS diff
                        FROM    sm_input
                        ),
                        sm_exp AS
                        (
                        SELECT  head, x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
                        FROM    sm_diff
                        ),
                        softmax AS
                        (
                        SELECT  head, x, y AS place, e / SUM(e) OVER (PARTITION BY head, x) AS value
                        FROM    sm_exp
                        ),
                        attention AS
                        (
                        SELECT  place, ARRAY_AGG(value ORDER BY head * 64 + ordinality)::VECTOR(768) AS values
                        FROM    (
                                SELECT  head, x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * heads.v) AS values
                                FROM    softmax
                                JOIN    heads
                                USING   (head, place)
                                GROUP BY
                                        head, x
                                ) q
                        CROSS JOIN LATERAL
                                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
                        GROUP BY
                                place
                        ),
                        mha AS
                        (
                        SELECT  place, w.values + c_proj_b.values + previous.values AS values
                        FROM    (
                                SELECT  attention.place, ARRAY_AGG(INNER_PRODUCT(attention.values, c_proj_w.values) ORDER BY c_proj_w.place)::VECTOR(768) AS values
                                FROM    attention
                                CROSS JOIN
                                        c_proj_w
                                GROUP BY
                                        attention.place
                                ) w
                        CROSS JOIN
                                c_proj_b
                        JOIN    previous
                        USING   (place)
                        ),
                        ffn_norm AS
                        (
                        SELECT  place, agg.values * ln_2_g.values + ln_2_b.values AS values
                        FROM    (
                                SELECT  place, norm.values
                                FROM    mha
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  AVG(value) AS mean,
                                                VAR_POP(value) AS variance
                                        FROM    UNNEST(values::REAL[]) value
                                        ) agg
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                        ) norm
                                ) agg
                        CROSS JOIN
                                ln_2_b
                        CROSS JOIN
                                ln_2_g
                        ),
                        ffn_a AS
                        (
                        SELECT  gelu.place, gelu.values
                        FROM    (
                                SELECT  place, w.values + mlp_c_fc_b.values AS values
                                FROM    (
                                        SELECT  ffn_norm.place, ARRAY_AGG(INNER_PRODUCT(ffn_norm.values, mlp_c_fc_w.values) ORDER BY mlp_c_fc_w.place)::VECTOR(3072) AS values
                                        FROM    ffn_norm
                                        CROSS JOIN
                                                mlp_c_fc_w
                                        GROUP BY
                                                ffn_norm.place
                                        ) w
                                CROSS JOIN
                                        mlp_c_fc_b
                                ) v
                        CROSS JOIN LATERAL
                                (
                                SELECT  place, ARRAY_AGG(0.5 * value * (1 + TANH(0.797884560802 * (value + 0.044715 * value*value*value))) ORDER BY ordinality)::VECTOR(3072) AS values
                                FROM    UNNEST(values::REAL[]) WITH ORDINALITY n (value, ordinality)
                                GROUP BY
                                        place
                                ) gelu
                        ),
                        ffn AS
                        (
                        SELECT  place, w.values + mlp_c_proj_b.values + mha.values AS values
                        FROM    (
                                SELECT  ffn_a.place, ARRAY_AGG(INNER_PRODUCT(ffn_a.values, mlp_c_proj_w.values) ORDER BY mlp_c_proj_w.place)::VECTOR(768) AS values
                                FROM    ffn_a
                                CROSS JOIN
                                        mlp_c_proj_w
                                GROUP BY
                                        ffn_a.place
                                ) w
                        CROSS JOIN
                                mlp_c_proj_b
                        JOIN    mha
                        USING   (place)
                        )
                SELECT  *
                FROM    ffn
                ) transformed_layer
        )
        ),
        block_output AS
        (
        SELECT  *
        FROM    hparams
        JOIN    transform
        ON      transform.block = n_block
        ),
        ln_f AS
        (
        SELECT  place, norm.values * ln_f_g.values + ln_f_b.values AS values
        FROM    block_output
        CROSS JOIN LATERAL
                (
                SELECT  AVG(value) AS mean,
                        VAR_POP(value) AS variance
                FROM    UNNEST(values::REAL[]) AS n(value)
                ) agg
        CROSS JOIN LATERAL
                (
                SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n (value, ordinality)
                ) norm
        CROSS JOIN
                ln_f_b
        CROSS JOIN
                ln_f_g
        )
SELECT  place,
        (SELECT STRING_AGG(TO_CHAR(n, 'S0.000'), ' ') || ' …' FROM UNNEST((values::REAL[])[:10]) AS n) AS q
FROM    ln_f
378876fdfdedf326e8861f0f96c15bc7.png

Это вывод модели.

Четвёртый вектор – это и есть фактическое представление следующего токена, предсказанного моделью. Нам просто нужно отобразить его обратно на токены.

Токены

У нас уже есть векторное представление (768-мерное), в котором, согласно модели, заключены семантика и грамматика наиболее вероятного продолжения промпта. Теперь нам нужно отобразить его обратно на токен.

Одна из первых операций, совершаемых моделью – это отображение токенов на их векторные представления. Эта операция делается при помощи матрицы wpe размером 50257×768. Той же самой матрицей нам потребуется воспользоваться, чтобы отобразить векторное представление обратно на токен.

Проблема в том, что строго обратное отображение выполнить невозможно: (вероятно) векторное представление не будет совпадать ни с одной из строк в матрице. Поэтому нам придётся найти токен, который «ближе всего» к векторному представлению.

Поскольку (как мы надеемся), в измерениях векторных представлений заключены некоторые семантические и грамматические аспекты токена, необходимо сопоставить их как можно точнее. Один из способов консолидировать «степень близости» каждого из измерений – вычислить скалярное произведение двух векторных представлений. Чем выше будет это значение, тем ближе токен к прогнозу.

Для этого умножим векторное представление на матрицу wte. В результате получим матрицу из одного столбца и 50257 строк. Каждое значение в этом результате будет произведением спрогнозированного вектора и векторного представления токена. Чем выше это число, тем вероятнее, что токен станет продолжением промпта.

Чтобы выбрать следующий токен, нужно просто преобразовать значения сходства в значения вероятности. Для этого нам пригодится старая добрая функция softmax (та самая, при помощи которой мы уже нормализовали веса внимания).

Почему стоит использовать softmax для работы с вероятностями?

Softmax – это удобное свойство аксиомы выбора Льюса. Оно таково: относительная вероятность двух вариантов не зависит от наличия прочих вероятностных вариантов. Если вариант A влвое вероятнее варианта B, то это соотношение не зависит от наличия или отсутствия других вариантов (хотя, конечно, абсолютные значения при этом могут измениться).

Вектор скалярных произведений («логит» в терминологии ИИ) содержит произвольные значения, которые могут не укладываться в какую-либо шкалу. Если балл A выше, чем балл B, то известно, что A вероятнее – а больше ничего не известно. Можно подкорректировать входные значения softmax так, как нам угодно, главное, соблюдать их порядок (более крупные значения остаются более крупными).

Распространённый подход, позволяющий этого добиться – нормализация значений путём вычитания наибольшего значения из каждого значения в множестве (так что наибольшее значение становится равным 0, а все остальные – отрицательным числам). Затем берём фиксированное количество наивысших значений (скажем, топ-5 или топ-10). Наконец, перед передачей всех значений softmax умножаем их на константу.

Количество наивысших значений, с которым мы работаем, обычно называется top_n, а множитель-константа (вернее, обратное ему число) называется «температурой» (Т). Чем выше температура, тем более гладкое распределение вероятностей у нас получится, и тем выше вероятность, что на место продолжения будет выбран не первый попавшийся токен.

Формула для расчёта вероятностей токенов

a6a685d634b1bb26d07ca1cb2cdda672.png

где scores — это множество из наивысших top_n баллов.

Почему этот показатель называется "температурой"

Функция softmax также называется распределение Больцмана. Она широко используется в физике. Среди прочего, на ней основана барометрическая формула, по которой рассчитывается изменение плотности воздуха в зависимости от высоты.

Как известно, тёплый воздух поднимается вверх и распределяется на сравнительно большой высоте. В горячем воздухе сравнительно высока вероятность, что молекула ль частых столкновений с другими молекулами заберётся на такую высоту, куда в ином случае бы не попала. В отличие от ситуации с более холодными температурами, плотность воздуха увеличивается на больших высотах и падает на уровне моря.

Сравните, как воздух движется при разных температурах:

e109ce6f14a9d797ac4ced0f00151a86.gif792fe2af01b49345c3ac0a6c8e08502a.gif

\Иллюстрация Доминика Форда, Bouncing Balls and the Boltzmann Distribution

Аналогично, в рассматриваемом нами примере, чем больше «температура», тем выше вероятность, что будет выбран не самый очевидный токен (разумеется, за счёт наиболее очевидных). Логический вывод становится менее предсказуемым и более «творческим».

Давайте выразим всё это на SQL. У нас был промпт «PostgreSQL is great». Вот первые 5 токенов, которые, в соответствии с этой моделью, являются наиболее вероятными продолжениями этой фразы. Показаны их вероятности при разной температуре:

WITH    RECURSIVE
        initial AS
        (
        SELECT  ARRAY[6307, 47701, 318, 1049] AS input
        ),
        hparams AS
        (
        SELECT  12 AS n_block,
                5 AS top_n,
                ARRAY_LENGTH(input, 1) AS n_seq
        FROM    initial
        ),
        embeddings AS
        (
        SELECT  place, values
        FROM    initial
        CROSS JOIN
                hparams
        CROSS JOIN LATERAL
                UNNEST(input) WITH ORDINALITY AS tokens (token, ordinality)
        CROSS JOIN LATERAL
                (
                SELECT  ordinality - 1 AS place
                ) o
        CROSS JOIN LATERAL
                (
                SELECT  wte.values + wpe.values AS values
                FROM    wte
                CROSS JOIN
                        wpe
                WHERE   wte.token = tokens.token
                        AND wpe.place = o.place
                ) embedding
        ),
        transform AS
        (
        SELECT  0 AS block, place, values
        FROM    embeddings
        UNION ALL
        (
        WITH    previous AS
                (
                SELECT  *
                FROM    transform
                )
        SELECT  block + 1 AS block, transformed_layer.*
        FROM    hparams
        CROSS JOIN LATERAL
                (
                SELECT  block
                FROM    previous
                WHERE   block < 12
                LIMIT   1
                ) q
        CROSS JOIN LATERAL
                (
                WITH    ln_2_b AS
                        (
                        SELECT  *
                        FROM    ln_2_b
                        WHERE   block = q.block
                        ),
                        ln_2_g AS
                        (
                        SELECT  *
                        FROM    ln_2_g
                        WHERE   block = q.block
                        ),
                        c_proj_w AS
                        (
                        SELECT  *
                        FROM    c_proj_w
                        WHERE   block = q.block
                        ),
                        c_proj_b AS
                        (
                        SELECT  *
                        FROM    c_proj_b
                        WHERE   block = q.block
                        ),
                        mlp_c_fc_w AS
                        (
                        SELECT  *
                        FROM    mlp_c_fc_w
                        WHERE   block = q.block
                        ),
                        mlp_c_fc_b AS
                        (
                        SELECT  *
                        FROM    mlp_c_fc_b
                        WHERE   block = q.block
                        ),
                        mlp_c_proj_w AS
                        (
                        SELECT  *
                        FROM    mlp_c_proj_w
                        WHERE   block = q.block
                        ),
                        mlp_c_proj_b AS
                        (
                        SELECT  *
                        FROM    mlp_c_proj_b
                        WHERE   block = q.block
                        ),
                        c_attn_w AS
                        (
                        SELECT  *
                        FROM    c_attn_w
                        WHERE   block = q.block
                        ),
                        c_attn_b AS
                        (
                        SELECT  *
                        FROM    c_attn_b
                        WHERE   block = q.block
                        ),
                        ln_1_g AS
                        (
                        SELECT  *
                        FROM    ln_1_g
                        WHERE   block = q.block
                        ),
                        ln_1_b AS
                        (
                        SELECT  *
                        FROM    ln_1_b
                        WHERE   block = q.block
                        ),
                        mha_norm AS
                        (
                        SELECT  place, mm.values + c_attn_b.values AS values
                        FROM    (
                                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                                FROM    (
                                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                                        FROM    (
                                                SELECT  place, norm.values
                                                FROM    previous
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  AVG(value) AS mean,
                                                                VAR_POP(value) AS variance
                                                        FROM    UNNEST(values::REAL[]) value
                                                        ) agg
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                                        ) norm
                                                ) agg
                                        CROSS JOIN
                                                ln_1_b
                                        CROSS JOIN
                                                ln_1_g
                                        ) layer_norm
                                CROSS JOIN
                                        c_attn_w
                                GROUP BY
                                        place
                                ) mm
                        CROSS JOIN
                                c_attn_b
                        ),
                        heads AS
                        (
                        SELECT  place, head,
                                (values::REAL[])[(head * 64 + 1):(head * 64 + 64)]::VECTOR(64) AS q,
                                (values::REAL[])[(head * 64 + 1 + 768):(head * 64 + 64 + 768)]::VECTOR(64) AS k,
                                (values::REAL[])[(head * 64 + 1 + 1536):(head * 64 + 64 + 1536)]::VECTOR(64) AS v
                        FROM    mha_norm
                        CROSS JOIN
                                GENERATE_SERIES(0, 11) head
                        ),
                        sm_input AS
                        (
                        SELECT  head, h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
                        FROM    heads h1
                        JOIN    heads h2
                        USING   (head)
                        ),
                        sm_diff AS
                        (
                        SELECT  head, x, y, value - MAX(value) OVER (PARTITION BY head, x) AS diff
                        FROM    sm_input
                        ),
                        sm_exp AS
                        (
                        SELECT  head, x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
                        FROM    sm_diff
                        ),
                        softmax AS
                        (
                        SELECT  head, x, y AS place, e / SUM(e) OVER (PARTITION BY head, x) AS value
                        FROM    sm_exp
                        ),
                        attention AS
                        (
                        SELECT  place, ARRAY_AGG(value ORDER BY head * 64 + ordinality)::VECTOR(768) AS values
                        FROM    (
                                SELECT  head, x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * heads.v) AS values
                                FROM    softmax
                                JOIN    heads
                                USING   (head, place)
                                GROUP BY
                                        head, x
                                ) q
                        CROSS JOIN LATERAL
                                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
                        GROUP BY
                                place
                        ),
                        mha AS
                        (
                        SELECT  place, w.values + c_proj_b.values + previous.values AS values
                        FROM    (
                                SELECT  attention.place, ARRAY_AGG(INNER_PRODUCT(attention.values, c_proj_w.values) ORDER BY c_proj_w.place)::VECTOR(768) AS values
                                FROM    attention
                                CROSS JOIN
                                        c_proj_w
                                GROUP BY
                                        attention.place
                                ) w
                        CROSS JOIN
                                c_proj_b
                        JOIN    previous
                        USING   (place)
                        ),
                        ffn_norm AS
                        (
                        SELECT  place, agg.values * ln_2_g.values + ln_2_b.values AS values
                        FROM    (
                                SELECT  place, norm.values
                                FROM    mha
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  AVG(value) AS mean,
                                                VAR_POP(value) AS variance
                                        FROM    UNNEST(values::REAL[]) value
                                        ) agg
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                        ) norm
                                ) agg
                        CROSS JOIN
                                ln_2_b
                        CROSS JOIN
                                ln_2_g
                        ),
                        ffn_a AS
                        (
                        SELECT  gelu.place, gelu.values
                        FROM    (
                                SELECT  place, w.values + mlp_c_fc_b.values AS values
                                FROM    (
                                        SELECT  ffn_norm.place, ARRAY_AGG(INNER_PRODUCT(ffn_norm.values, mlp_c_fc_w.values) ORDER BY mlp_c_fc_w.place)::VECTOR(3072) AS values
                                        FROM    ffn_norm
                                        CROSS JOIN
                                                mlp_c_fc_w
                                        GROUP BY
                                                ffn_norm.place
                                        ) w
                                CROSS JOIN
                                        mlp_c_fc_b
                                ) v
                        CROSS JOIN LATERAL
                                (
                                SELECT  place, ARRAY_AGG(0.5 * value * (1 + TANH(0.797884560802 * (value + 0.044715 * value*value*value))) ORDER BY ordinality)::VECTOR(3072) AS values
                                FROM    UNNEST(values::REAL[]) WITH ORDINALITY n (value, ordinality)
                                GROUP BY
                                        place
                                ) gelu
                        ),
                        ffn AS
                        (
                        SELECT  place, w.values + mlp_c_proj_b.values + mha.values AS values
                        FROM    (
                                SELECT  ffn_a.place, ARRAY_AGG(INNER_PRODUCT(ffn_a.values, mlp_c_proj_w.values) ORDER BY mlp_c_proj_w.place)::VECTOR(768) AS values
                                FROM    ffn_a
                                CROSS JOIN
                                        mlp_c_proj_w
                                GROUP BY
                                        ffn_a.place
                                ) w
                        CROSS JOIN
                                mlp_c_proj_b
                        JOIN    mha
                        USING   (place)
                        )
                SELECT  *
                FROM    ffn
                ) transformed_layer
        )
        ),
        block_output AS
        (
        SELECT  *
        FROM    hparams
        JOIN    transform
        ON      transform.block = n_block
        ),
        ln_f AS
        (
        SELECT  place, norm.values * ln_f_g.values + ln_f_b.values AS values
        FROM    block_output
        CROSS JOIN LATERAL
                (
                SELECT  AVG(value) AS mean,
                        VAR_POP(value) AS variance
                FROM    UNNEST(values::REAL[]) AS n(value)
                ) agg
        CROSS JOIN LATERAL
                (
                SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n (value, ordinality)
                ) norm
        CROSS JOIN
                ln_f_b
        CROSS JOIN
                ln_f_g
        ),
        logits AS
        (
        SELECT  logits.*
        FROM    hparams
        CROSS JOIN LATERAL
                (
                SELECT  token, INNER_PRODUCT(ln_f.values, wte.values) AS value
                FROM    ln_f
                CROSS JOIN
                        wte
                WHERE   ln_f.place = n_seq - 1
                ORDER BY
                        value DESC
                LIMIT   (top_n)
                ) logits
        ),
        temperatures (temperature) AS
        (
        VALUES
        (0.5),
        (1),
        (2)
        ),
        tokens AS
        (
        SELECT  token, value, softmax, temperature
        FROM    temperatures
        CROSS JOIN LATERAL
                (
                SELECT  *, (e / SUM(e) OVER ()) AS softmax
                FROM    (
                        SELECT  *,
                                (value - MAX(value) OVER ()) / temperature AS diff
                        FROM    logits
                        ) exp_x
                CROSS JOIN LATERAL
                        (
                        SELECT  CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
                        ) exp
                ) q
        )
SELECT  token,
        cluster,
        TO_CHAR(t1.value, 'S00.000') AS score,
        TO_CHAR(t1.softmax, '0.00') AS "temperature = 0.5",
        TO_CHAR(t2.softmax, '0.00') AS "temperature = 1",
        TO_CHAR(t3.softmax, '0.00') AS "temperature = 2"
FROM    (
        SELECT  *
        FROM    tokens
        WHERE   temperature = 0.5
        ) t1
JOIN    (
        SELECT  *
        FROM    tokens
        WHERE   temperature = 1
        ) t2
USING   (token)
JOIN    (
        SELECT  *
        FROM    tokens
        WHERE   temperature = 2
        ) t3
USING   (token)
JOIN    tokenizer
USING   (token)
39b00c06e909aab6901bceb4228137b2.png

Логический вывод

Вот мы и готовы перейти к реальному логическому выводу: запустить модель, выбрать токен в соответствии с полученной вероятностью, добавить токен к промпту и повторять этот процесс до тех пор, пока не будет сгенерировано достаточное количество токенов.

Как было показано выше, сама большая языковая модель является детерминированной. Это просто последовательные перемножения матриц и другие математические операции, производимые над заранее определёнными константами. Пока промпт и такие гиперпараметры как температура и топ-N остаются неизменными, вывод также остаётся неизменным.

Единственный недетерминированный процесс в данном случае – это выбор токена. В нём заложена (переменная) степень случайности. Вот почему чатботы, основанные на GPT, могут давать разные ответы на один и тот же промпт.

Воспользуемся в качестве промпта фразой «Happy New Year! I wish» и прикажем модели сгенерировать 10 новых токенов для этого промпта. Температуру установим в 2, а top_n в 5.

У меня на компьютере на этот запрос уходит 2 минуты и 44 секунды. Вот такой вывод получается:

SELECT SETSEED(0.20231231);
 
WITH    RECURSIVE
        input AS
        (
        SELECT  'Happy New Year! I wish you' AS prompt,
                10 AS threshold,
                2 AS temperature,
                1 AS top_n
        ),
        clusters AS
        (
        SELECT  part_position, bpe.*
        FROM    input
        CROSS JOIN LATERAL
                REGEXP_MATCHES(prompt, '''s|''t|''re|''ve|''m|''ll|''d| ?\w+| ?\d+| ?[^\s\w\d]+|\s+(?!\S)|\s+', 'g') WITH ORDINALITY AS rm (part, part_position)
        CROSS JOIN LATERAL
                (
                WITH    RECURSIVE
                        bpe AS
                        (
                        SELECT  (n + 1)::BIGINT AS position, character, TRUE AS continue
                        FROM    CONVERT_TO(part[1], 'UTF-8') AS bytes
                        CROSS JOIN LATERAL
                                GENERATE_SERIES(0, LENGTH(bytes) - 1) AS n
                        JOIN    encoder
                        ON      byte = GET_BYTE(bytes, n)
                        UNION ALL
                        (
                        WITH    RECURSIVE
                                base AS
                                (
                                SELECT  *
                                FROM    bpe
                                WHERE   continue
                                ),
                                bn AS
                                (
                                SELECT  ROW_NUMBER() OVER (ORDER BY position) AS position,
                                        continue,
                                        character,
                                        character || LEAD(character) OVER (ORDER BY position) AS cluster
                                FROM    base
                                ),
                                top_rank AS
                                (
                                SELECT  tokenizer.*
                                FROM    bn
                                CROSS JOIN LATERAL
                                        (
                                        SELECT  *
                                        FROM    tokenizer
                                        WHERE   tokenizer.cluster = bn.cluster
                                        LIMIT   1
                                        ) tokenizer
                                ORDER BY
                                        token
                                LIMIT   1
                                ),
                                breaks AS
                                (
                                SELECT  0::BIGINT AS position, 1 AS length
                                UNION ALL
                                SELECT  bn.position,
                                        CASE WHEN token IS NULL THEN 1 ELSE 2 END
                                FROM    breaks
                                JOIN    bn
                                ON      bn.position = breaks.position + length
                                LEFT JOIN
                                        top_rank
                                USING   (cluster)
                                )
                        SELECT  position, character, token IS NOT NULL
                        FROM    breaks
                        LEFT JOIN
                                top_rank
                        ON      1 = 1
                        CROSS JOIN LATERAL
                                (
                                SELECT  STRING_AGG(character, '' ORDER BY position) AS character
                                FROM    bn
                                WHERE   bn.position >= breaks.position
                                        AND bn.position < breaks.position + length
                                ) bn
                        WHERE   position > 0
                        )
                        )
                SELECT  position, character AS cluster
                FROM    bpe
                WHERE   NOT continue
                ) bpe
        ),
        tokens AS
        (
        SELECT  ARRAY_AGG(token ORDER BY part_position, position) AS input
        FROM    clusters
        JOIN    tokenizer
        USING   (cluster)
        ),
        gpt AS
        (
        SELECT  input, ARRAY_LENGTH(input, 1) AS original_length
        FROM    tokens
        UNION ALL
        SELECT  input || next_token.token, original_length
        FROM    gpt
        CROSS JOIN
                input
        CROSS JOIN LATERAL
                (
                WITH    RECURSIVE
                        hparams AS
                        (
                        SELECT  ARRAY_LENGTH(input, 1) AS n_seq,
                                12 AS n_block
                        ),
                        embeddings AS
                        (
                        SELECT  place, values
                        FROM    hparams
                        CROSS JOIN LATERAL
                                UNNEST(input) WITH ORDINALITY AS tokens (token, ordinality)
                        CROSS JOIN LATERAL
                                (
                                SELECT  ordinality - 1 AS place
                                ) o
                        CROSS JOIN LATERAL
                                (
                                SELECT  wte.values + wpe.values AS values
                                FROM    wte
                                CROSS JOIN
                                        wpe
                                WHERE   wte.token = tokens.token
                                        AND wpe.place = o.place
                                ) embedding
                        ),
                        transform AS
                        (
                        SELECT  0 AS block, place, values
                        FROM    embeddings
                        UNION ALL
                        (
                        WITH    previous AS
                                (
                                SELECT  *
                                FROM    transform
                                )
                        SELECT  block + 1 AS block, transformed_layer.*
                        FROM    hparams
                        CROSS JOIN LATERAL
                                (
                                SELECT  block
                                FROM    previous
                                WHERE   block < 12
                                LIMIT   1
                                ) q
                        CROSS JOIN LATERAL
                                (
                                WITH    ln_2_b AS
                                        (
                                        SELECT  *
                                        FROM    ln_2_b
                                        WHERE   block = q.block
                                        ),
                                        ln_2_g AS
                                        (
                                        SELECT  *
                                        FROM    ln_2_g
                                        WHERE   block = q.block
                                        ),
                                        c_proj_w AS
                                        (
                                        SELECT  *
                                        FROM    c_proj_w
                                        WHERE   block = q.block
                                        ),
                                        c_proj_b AS
                                        (
                                        SELECT  *
                                        FROM    c_proj_b
                                        WHERE   block = q.block
                                        ),
                                        mlp_c_fc_w AS
                                        (
                                        SELECT  *
                                        FROM    mlp_c_fc_w
                                        WHERE   block = q.block
                                        ),
                                        mlp_c_fc_b AS
                                        (
                                        SELECT  *
                                        FROM    mlp_c_fc_b
                                        WHERE   block = q.block
                                        ),
                                        mlp_c_proj_w AS
                                        (
                                        SELECT  *
                                        FROM    mlp_c_proj_w
                                        WHERE   block = q.block
                                        ),
                                        mlp_c_proj_b AS
                                        (
                                        SELECT  *
                                        FROM    mlp_c_proj_b
                                        WHERE   block = q.block
                                        ),
                                        c_attn_w AS
                                        (
                                        SELECT  *
                                        FROM    c_attn_w
                                        WHERE   block = q.block
                                        ),
                                        c_attn_b AS
                                        (
                                        SELECT  *
                                        FROM    c_attn_b
                                        WHERE   block = q.block
                                        ),
                                        ln_1_g AS
                                        (
                                        SELECT  *
                                        FROM    ln_1_g
                                        WHERE   block = q.block
                                        ),
                                        ln_1_b AS
                                        (
                                        SELECT  *
                                        FROM    ln_1_b
                                        WHERE   block = q.block
                                        ),
                                        mha_norm AS
                                        (
                                        SELECT  place, mm.values + c_attn_b.values AS values
                                        FROM    (
                                                SELECT  place, ARRAY_AGG(INNER_PRODUCT(c_attn_w.values, layer_norm.values) ORDER BY y)::VECTOR(2304) AS values
                                                FROM    (
                                                        SELECT  place, agg.values * ln_1_g.values + ln_1_b.values AS values
                                                        FROM    (
                                                                SELECT  place, norm.values
                                                                FROM    previous
                                                                CROSS JOIN LATERAL
                                                                        (
                                                                        SELECT  AVG(value) AS mean,
                                                                                VAR_POP(value) AS variance
                                                                        FROM    UNNEST(values::REAL[]) value
                                                                        ) agg
                                                                CROSS JOIN LATERAL
                                                                        (
                                                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                                                        ) norm
                                                                ) agg
                                                        CROSS JOIN
                                                                ln_1_b
                                                        CROSS JOIN
                                                                ln_1_g
                                                        ) layer_norm
                                                CROSS JOIN
                                                        c_attn_w
                                                GROUP BY
                                                        place
                                                ) mm
                                        CROSS JOIN
                                                c_attn_b
                                        ),
                                        heads AS
                                        (
                                        SELECT  place, head,
                                                (values::REAL[])[(head * 64 + 1):(head * 64 + 64)]::VECTOR(64) AS q,
                                                (values::REAL[])[(head * 64 + 1 + 768):(head * 64 + 64 + 768)]::VECTOR(64) AS k,
                                                (values::REAL[])[(head * 64 + 1 + 1536):(head * 64 + 64 + 1536)]::VECTOR(64) AS v
                                        FROM    mha_norm
                                        CROSS JOIN
                                                GENERATE_SERIES(0, 11) head
                                        ),
                                        sm_input AS
                                        (
                                        SELECT  head, h1.place AS x, h2.place AS y, INNER_PRODUCT(h1.q, h2.k) / 8 + CASE WHEN h2.place > h1.place THEN -1E10 ELSE 0 END AS value
                                        FROM    heads h1
                                        JOIN    heads h2
                                        USING   (head)
                                        ),
                                        sm_diff AS
                                        (
                                        SELECT  head, x, y, value - MAX(value) OVER (PARTITION BY head, x) AS diff
                                        FROM    sm_input
                                        ),
                                        sm_exp AS
                                        (
                                        SELECT  head, x, y, CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
                                        FROM    sm_diff
                                        ),
                                        softmax AS
                                        (
                                        SELECT  head, x, y AS place, e / SUM(e) OVER (PARTITION BY head, x) AS value
                                        FROM    sm_exp
                                        ),
                                        attention AS
                                        (
                                        SELECT  place, ARRAY_AGG(value ORDER BY head * 64 + ordinality)::VECTOR(768) AS values
                                        FROM    (
                                                SELECT  head, x AS place, SUM(ARRAY_FILL(softmax.value, ARRAY[64])::VECTOR(64) * heads.v) AS values
                                                FROM    softmax
                                                JOIN    heads
                                                USING   (head, place)
                                                GROUP BY
                                                        head, x
                                                ) q
                                        CROSS JOIN LATERAL
                                                UNNEST(values::REAL[]) WITH ORDINALITY v (value, ordinality)
                                        GROUP BY
                                                place
                                        ),
                                        mha AS
                                        (
                                        SELECT  place, w.values + c_proj_b.values + previous.values AS values
                                        FROM    (
                                                SELECT  attention.place, ARRAY_AGG(INNER_PRODUCT(attention.values, c_proj_w.values) ORDER BY c_proj_w.place)::VECTOR(768) AS values
                                                FROM    attention
                                                CROSS JOIN
                                                        c_proj_w
                                                GROUP BY
                                                        attention.place
                                                ) w
                                        CROSS JOIN
                                                c_proj_b
                                        JOIN    previous
                                        USING   (place)
                                        ),
                                        ffn_norm AS
                                        (
                                        SELECT  place, agg.values * ln_2_g.values + ln_2_b.values AS values
                                        FROM    (
                                                SELECT  place, norm.values
                                                FROM    mha
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  AVG(value) AS mean,
                                                                VAR_POP(value) AS variance
                                                        FROM    UNNEST(values::REAL[]) value
                                                        ) agg
                                                CROSS JOIN LATERAL
                                                        (
                                                        SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                                        FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n(value, ordinality)
                                                        ) norm
                                                ) agg
                                        CROSS JOIN
                                                ln_2_b
                                        CROSS JOIN
                                                ln_2_g
                                        ),
                                        ffn_a AS
                                        (
                                        SELECT  gelu.place, gelu.values
                                        FROM    (
                                                SELECT  place, w.values + mlp_c_fc_b.values AS values
                                                FROM    (
                                                        SELECT  ffn_norm.place, ARRAY_AGG(INNER_PRODUCT(ffn_norm.values, mlp_c_fc_w.values) ORDER BY mlp_c_fc_w.place)::VECTOR(3072) AS values
                                                        FROM    ffn_norm
                                                        CROSS JOIN
                                                                mlp_c_fc_w
                                                        GROUP BY
                                                                ffn_norm.place
                                                        ) w
                                                CROSS JOIN
                                                        mlp_c_fc_b
                                                ) v
                                        CROSS JOIN LATERAL
                                                (
                                                SELECT  place, ARRAY_AGG(0.5 * value * (1 + TANH(0.797884560802 * (value + 0.044715 * value*value*value))) ORDER BY ordinality)::VECTOR(3072) AS values
                                                FROM    UNNEST(values::REAL[]) WITH ORDINALITY n (value, ordinality)
                                                GROUP BY
                                                        place
                                                ) gelu
                                        ),
                                        ffn AS
                                        (
                                        SELECT  place, w.values + mlp_c_proj_b.values + mha.values AS values
                                        FROM    (
                                                SELECT  ffn_a.place, ARRAY_AGG(INNER_PRODUCT(ffn_a.values, mlp_c_proj_w.values) ORDER BY mlp_c_proj_w.place)::VECTOR(768) AS values
                                                FROM    ffn_a
                                                CROSS JOIN
                                                        mlp_c_proj_w
                                                GROUP BY
                                                        ffn_a.place
                                                ) w
                                        CROSS JOIN
                                                mlp_c_proj_b
                                        JOIN    mha
                                        USING   (place)
                                        )
                                SELECT  *
                                FROM    ffn
                                ) transformed_layer
                        )
                        ),
                        block_output AS
                        (
                        SELECT  *
                        FROM    hparams
                        JOIN    transform
                        ON      transform.block = n_block
                        ),
                        ln_f AS
                        (
                        SELECT  place, norm.values * ln_f_g.values + ln_f_b.values AS values
                        FROM    block_output
                        CROSS JOIN LATERAL
                                (
                                SELECT  AVG(value) AS mean,
                                        VAR_POP(value) AS variance
                                FROM    UNNEST(values::REAL[]) AS n(value)
                                ) agg
                        CROSS JOIN LATERAL
                                (
                                SELECT  ARRAY_AGG((value - mean) / SQRT(variance + 1E-5) ORDER BY ordinality)::VECTOR(768) AS values
                                FROM    UNNEST(values::REAL[]) WITH ORDINALITY AS n (value, ordinality)
                                ) norm
                        CROSS JOIN
                                ln_f_b
                        CROSS JOIN
                                ln_f_g
                        ),
                        logits AS
                        (
                        SELECT  token, INNER_PRODUCT(ln_f.values, wte.values) AS value
                        FROM    hparams
                        JOIN    ln_f
                        ON      ln_f.place = n_seq - 1
                        CROSS JOIN
                                wte
                        ORDER BY
                                value DESC
                        LIMIT   (top_n)
                        ),
                        tokens AS
                        (
                        SELECT  token,
                                high - softmax AS low,
                                high
                        FROM    (
                                SELECT  *,
                                        SUM(softmax) OVER (ORDER BY softmax) AS high
                                FROM    (
                                        SELECT  *, (e / SUM(e) OVER ()) AS softmax
                                        FROM    (
                                                SELECT  *,
                                                        (value - MAX(value) OVER ()) / temperature AS diff
                                                FROM    logits
                                                ) exp_x
                                        CROSS JOIN LATERAL
                                                (
                                                SELECT  CASE WHEN diff < -745.13 THEN 0 ELSE EXP(diff) END AS e
                                                ) exp
                                        ) q
                                ) q
                        ),
                        next_token AS
                        (
                        SELECT  *
                        FROM    (
                                SELECT  RANDOM() AS rnd
                                ) r
                        CROSS JOIN LATERAL
                                (
                                SELECT  *
                                FROM    tokens
                                WHERE   rnd >= low
                                        AND rnd < high
                                ) nt
                        )
                SELECT  *
                FROM    next_token
                ) next_token
        WHERE   ARRAY_LENGTH(input, 1) < original_length + threshold
                AND next_token.token <> 50256
        ),
        output AS
        (
        SELECT  CONVERT_FROM(STRING_AGG(SET_BYTE('\x00', 0, byte), '' ORDER BY position), 'UTF8') AS response
        FROM    (
                SELECT  STRING_AGG(cluster, '' ORDER BY ordinality) AS response
                FROM    input
                JOIN    gpt
                ON      ARRAY_LENGTH(input, 1) = original_length + threshold
                CROSS JOIN LATERAL
                        UNNEST(input) WITH ORDINALITY n (token, ordinality)
                JOIN    tokenizer
                USING   (token)
                ) q
        CROSS JOIN LATERAL
                STRING_TO_TABLE(response, NULL) WITH ORDINALITY n (character, position)
        JOIN    encoder
        USING   (character)
        )
SELECT  *
FROM    output
1a00f841de73f9bb1baf6ce719e8ae7d.png

ИИ со своей работой справился отлично. Я искренне желаю вам всего самого лучшего в наступившем году!

Все запросы и код для установки выложены в следующем репозитории на GitHub: quassnoi/explain-extended-2024.

Источник

  • 08.08.25 11:10 elizabethrush89

    God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 08.08.25 11:10 elizabethrush89

    God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 08.08.25 13:38 Annette_Phillips

    A few months ago, I fell for what I thought was a legitimate staking platform. It looked so professional, had fake reviews, and everything seemed real. I transferred a significant amount of ETH into it, only to find out later that it was a complete scam. The site vanished, support stopped replying, and the wallet address I sent my funds to started moving the crypto across multiple chains. I felt completely helpless. I spent days blaming myself and trying to accept the loss, but part of me wasn’t ready to give up. That’s when someone recommended Asset-Resolute. I didn’t expect much, but I reached out and explained everything. To my surprise, they actually listened and started investigating right away. They traced my stolen funds through several wallets and gave me a full report of where it had moved. I honestly didn’t understand most of the technical details, but they broke it down clearly and kept me updated throughout the process. After some coordination and effort, they managed to recover a portion of my crypto. I never thought I’d see anything again, so getting anything back felt like a miracle. I just wanted to share this in case someone else out there is feeling the same panic and regret I felt. It’s not always the end. Reach out to Asset-Resolute at [email protected]. They gave me hope when I had none left.

  • 08.08.25 14:40 traviscluster

    The shock of losing a significant amount of cryptocurrency was immense. It felt like hitting rock bottom. Every avenue I explored to recover my funds proved fruitless. I was sinking deeper into despair, convinced my money was gone forever. Then, a beacon of hope appeared in the form of Sylvester Bryant. His intervention was a complete turning point. Sylvester’s expertise was truly remarkable. He managed to recover a substantial sum of over $780,000 in USDT. This alone was a life-changing achievement. But his assistance didn't stop there. He went above and beyond, actively helping me to reclaim even more of my lost assets. Throughout the entire intricate process, he maintained constant communication. He ensured I was fully informed at every stage. The entire operation was conducted with the utmost transparency and integrity. There were no hidden fees or deceptive practices. My funds were returned directly to my crypto wallet. The transaction was smooth and completely problem-free. I can honestly say Sylvester Bryant is the most trustworthy recovery agent I have ever encountered. I strongly advise anyone in a similar predicament to reach out to him immediately. You can connect with Sylvester via email at yt7cracker@gmail. com. Alternatively, you can contact him on WhatsApp at +1 (512) 577-7957. Do not allow yourself to succumb to despair. Sylvester possesses the skill and dedication to retrieve your stolen funds. He can make a difference for you, just as he did for me. Your financial recovery is possible. Sylvester Bryant is the key to getting your money back.

  • 08.08.25 16:16 briannawright679

    Jasmine Lopez, renowned for her expertise in recovering stolen cryptocurrency, especially specializing in USDT, provided invaluable assistance in a scenario where a significant sum of $571,000 was taken from me. I had the privilege of witnessing her exceptional skills firsthand and I am truly thankful for her prompt efforts, which led to the successful recovery of the entire amount in just a span of 72 hours. Miss Lopez's swift reactions and efficient tactics not only resolved my financial troubles but also alleviated much of the stress on my mental well-being. You can reach out to her immediately and get connected with email at [email protected] also can be contacted on WhatsApp at +44 (736) 644- 5035, She is an expert in the field most especially in recovering lost digital assets.

  • 08.08.25 18:16 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 08.08.25 18:16 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 08.08.25 22:01 abeggnatalie

    Throughout the ordeal, we were provided with frequent updates and full transparency. In just 10 days, TECHY FORCE CYBER RETRIEVAL successfully retrieved $191000 of the lost funds. Beyond the monetary recovery, they provided us with a comprehensive post-incident report, enabling us to strengthen our cybersecurity protocols and conduct risk training for partner organizations. Their work was nothing short of extraordinary. TECHY FORCE CYBER RETRIEVAL did not just restore stolen assets; they restored our operational integrity, capacity, and sense of direction. WhatsApp  (+156 172 636 97)     Mail   (support (At) techy forcecy berretrieval (Dot)com)      Telegram  (At) Techcyberforc

  • 08.08.25 22:02 abeggnatalie

    As the finance director of a humanitarian NGO, I have always exercised extreme diligence in how we handle our funding. So when I received an email from what appeared to be a prestigious global crypto foundation offering $500000 in matching grants for blockchain-based aid initiatives, I took notice. The proposal seemed credible, complete with polished branding, legal agreements, and referenced success stories. The only stipulation was an activation requirement: we had to transfer $250000 in USDT to initiate the grant process. After extensive internal consultation and document verification, we proceeded. The promise of doubling our operational funding could significantly amplify our outreach programs in conflict-affected regions. Then everything went silent. Follow-up emails were ignored, the contact number was disconnected, and within days, the foundation’s website vanished. The realization hit us hard; we had been defrauded. The emotional and reputational toll was immense. I felt personally culpable for having signed off on such a monumental error. It jeopardized not only our funding but also trust among our stakeholders. We turned to TECHY FORCE CYBER RETRIEVAL. Their response was immediate and professional. Within hours, we were assigned a dedicated case analyst. Their team operated with surgical precision. We learned we weren’t the only victims. This was a coordinated scam ring targeting nonprofits using highly customized and persuasive tactics. TECHY FORCE CYBER RETRIEVAL’s digital forensics experts tracked the stolen USDT through a labyrinth of six wallet addresses across various jurisdictions. Using advanced blockchain analytics, real-time monitoring software, and legal liaisons, they mapped the laundering trail with remarkable accuracy. They also coordinated directly with international regulators and the legal department of the real crypto foundation, which was alarmed to discover its identity had been fraudulently used. The recovery effort was rigorous and multifaceted, involving compliance teams at multiple exchanges, forensic IP tracing, and the audit of smart contracts linked to fraudulent addresses. Throughout the ordeal, we were provided with frequent updates and full transparency. In just 10 days, TECHY FORCE CYBER RETRIEVAL successfully retrieved $191000 of the lost funds. Beyond the monetary recovery, they provided us with a comprehensive post-incident report, enabling us to strengthen our cybersecurity protocols and conduct risk training for partner organizations. Their work was nothing short of extraordinary. TECHY FORCE CYBER RETRIEVAL did not just restore stolen assets; they restored our operational integrity, capacity, and sense of direction. WhatsApp  (+156 172 636 97)     Mail   (support (At) techy forcecy berretrieval (Dot)com)      Telegram  (At) Techcyberforc

  • 09.08.25 09:39 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 09.08.25 09:39 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 09.08.25 14:24 lisa111

    Recovering lost money from scams calls for cooperation. Recovery professionals locate stolen funds by applying their expertise. Experts in law offer advice on the best course of action. This helps you recoup your investment. Scams teach us to be aware of the warning indications. Promises of large, rapid riches with minimal risk should be avoided. They may want you to make an investment right away. Get advice from Marie on how to prevent frauds in the future. She can help you recover your missing bitcoin as well. You can reach her at [email protected] or @Marie_consultancy on Telegram. Additionally, you can utilize WhatsApp at +1 7127594675.

  • 09.08.25 16:31 vallatjosette

    My Wild Ride: How a Wizard Hilton Cyber Tech Helped Me Recover Lost Bitcoin of 103,000 It all started when I had the misfortune of misplacing the digital wallet containing my substantial cryptocurrency holdings - a staggering 103,000 bitcoins. I had amassed this sizable fortune over the years through strategic investments and careful trading, but in a moment of carelessness, the private keys to my digital wallet had vanished. Panic set in as I grappled with the gravity of my situation - those bitcoins, worth millions at the time, were effectively lost to the ether, inaccessible and seemingly unrecoverable. Just when I had resigned myself to the devastating financial blow, a chance encounter with a self-proclaimed "digital wizard" Wizard Hilton Cyber Tech presented a glimmer of hope. This enigmatic individual, who operated in the shadowy corners of the crypto underworld, claimed to possess the arcane knowledge and skills necessary to retrieve my lost digital wealth. Intrigued and desperate, I put my skepticism aside and entrusted this mysterious wizard with the task. What followed was a wild, suspenseful journey into the depths of blockchain technology and cryptography, as the wizard navigated complex algorithms, exploited vulnerabilities, and pieced together the puzzle of my missing private keys. After days of intense effort, Wizard Hilton Cyber Tech emerged triumphant, having successfully recovered the entirety of my 103,000 bitcoin stash. The sheer relief and elation I felt in that moment was indescribable, as I regained access to my small digital fortune and was able to secure my financial future. This harrowing experience taught me the invaluable lesson of never taking the security of my cryptocurrency holdings for granted, and the importance of always maintaining the utmost diligence when it comes to protecting one's digital wealth. Email: wizardhiltoncybertech ( @ ) gmail (. ) com     OR support ( @ ) wizardhiltoncybertech (.) com WhatsApp number  +18737715701 Thanks.

  • 09.08.25 16:43 meghanmarkle1998

    I was scammed out $ 180,000 in BTC, but Spikes Assets Recovery came to my rescue. When my cryptocurrency investment fell into the wrong hands, I lost $115,330 in Bitcoin and was defrauded of my hard-earned money. I was on the verge of giving up after learning that the funds seemed unrecoverable, but then I came across a Google post about a legitimate recovery service called Spikes Assets Recovery, a firm specializing in stolen fund and crypto recovery. To my astonishment, after working with them, they successfully recovered the full $115,330 I had lost in BTC. I'm incredibly grateful and relieved that I was able to recover all of my missing Bitcoin. For inquiries or assistance with recovering your lost assets, you can contact Spikes Assets Recovery using the following details: Email: [email protected] Yahoo: [email protected]

  • 09.08.25 17:24 Sophie Pugh

    I would strongly love to recommend the services of the best team of recoveryhackers101. They are professional and very discreet in carrying out their jobs, they have the best customer service agents and satisfaction at heart. If you have any services you wish to contact them for, go on (recoveryhacker101@gmailcom), They help track and monitor your cheating partner's phone without his idea, clear or erase criminal records as well as repair a bad credit score, all social media hacks, Recovery lost Funds from scammers / Cryptocurrency / Binary / Forex / Recovery of Stolen Bitcoin and many others.

  • 10.08.25 03:53 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:53 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:54 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:54 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:54 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:54 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 03:55 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram user name (@RoseCyberHives) and email rosecryptoinvestment11@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram user name (@RoseCyberHives) rosecryptoinvestment11@ gmail. com

  • 10.08.25 04:55 lisared

    I thought losing £85,000 in Doge was the end until Agent Jasmine Lopez proved me wrong. With unmatched skill and persistence, she traced my stolen money, tracked down the scammers, and recovered a huge portion of my funds. If you’ve been scammed, don’t wait reach out to her at (recoveryfundprovider@gmail. com) WhatsApp at +{44 736-644-5035}. She could be the reason you get your money back.

  • 10.08.25 12:57 Marthaamahle

    My name is Martha, and I never thought I’d be the victim of a crypto scam, but life has a way of humbling you. A few months ago, I invested a huge chunk of my savings, decades of hard-earned money into what seemed like a promising crypto platform. The slick website, glowing testimonials, and a charming advisor convinced me to transfer 15 Bitcoin, into their wallet. Days later, the platform vanished, my funds were gone, and I was left with nothing but regret. Desperate, I turned to the internet, scouring forums and posts on X for help. That’s when I stumbled across Alpha Spy Nest, a shadowy group of cyber-recovery experts specializing in crypto fraud. Their reputation was murky but promising whispers of successful recoveries and a no-nonsense approach. I reached out through their encrypted portal, half-expecting a scam. Within hours, they responded, asking for details: transaction IDs, wallet addresses, and screenshots of my interactions with the scammers. Alpha Spy Nest operated in the dark corners of the web, using a mix of blockchain forensics, social engineering, and unconventional methods to track lost crypto. They traced my Bitcoin to a series of obfuscated wallets, likely controlled by an offshore syndicate. The team deployed advanced chain-analysis tools to follow the funds, pinpointing a weak link in the scammers’ network, a poorly secured exchange account in Eastern Europe.Over weeks, Alpha Spy Nest worked tirelessly, infiltrating the scammers’ digital trails. And then i received a message, we’ve got a lead. They’d recovered 12 of my 15 Bitcoin, transferred to a secure wallet they set up for me. The rest, they said, was likely gone forever, but I was overjoyed. My savings were partially restored, and I felt a weight lift. I never met them, but they gave me back hope and a lesson, the crypto world is a jungle, but sometimes, even in the dark, there are allies. Contact them via: website: www.alphaspynest.org, whatsapp: ‪+15132924878‬ , telegram: https://t.me/Alphaspynest

  • 10.08.25 14:33 tomcooper0147

    Losing €256,000 On trading was one of the worst moments of my life. I felt sick, helpless, and convinced I’d never see that money again. Every day that passed made the loss feel heavier. Then I found Agent Jasmine Lopez and everything changed. With relentless determination, she traced the stolen funds, tracked down the scammers, and recovered a huge portion of my losses in the space of a few days. Her skill and persistence gave me hope when I had none. Time is everything contact her now: [email protected] Whats//App at +{44 736-644-5035}.

  • 10.08.25 17:06 kevin

    I see a lot of recommendations online and it’s already obvious there are bad eggs online who will only add to your mystery. I can only recommend one and you can reach them via mail on (zattechrecovery AT gmailc0 m ) if you need help on recovering what you lost to scammers.

  • 10.08.25 18:13 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 10.08.25 18:13 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 10.08.25 20:35 Bartlevi

    Sylvester demonstrated truly exceptional skill. He successfully recovered over $610,000 in USDT. This was a momentous personal triumph. His support extended far beyond this initial success. He diligently assisted in reclaiming additional lost funds. Throughout this complex process, he maintained consistent contact. He kept me fully updated at every turn. The entire undertaking was conducted with utmost honesty. There were no concealed charges or dubious methods. My funds were directly returned to my crypto wallet. The transaction proceeded smoothly and without any issues. I can confidently state Sylvester Bryant is the most reliable recovery agent I have ever encountered. I strongly urge anyone facing a similar situation to contact him without delay. You may reach Sylvester by email at yt7cracker@gmail. com. You can also contact him on WhatsApp at +1 (512) 577-7957. Do not surrender to despair. Sylvester possesses the ability and commitment to retrieve your stolen funds. He can achieve this for you, as he did for me. Your financial recovery is attainable. Sylvester Bryant is your path to recovering your money.

  • 10.08.25 23:28 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 10.08.25 23:29 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 11.08.25 10:33 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 11.08.25 10:33 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 11.08.25 10:34 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 11.08.25 10:34 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 11.08.25 11:35 Dennisblaq

    Hi everyone, I'm an entrepreneur, and when I learned about cryptocurrencies and how they potentially triple my income, I was excited to invest. I hope this message finds the correct audience. On the internet, I came across someone who assured me of exceptionally significant returns on my investment. I was readily persuaded by this man's seeming wisdom. I quickly discovered that this cryptocurrency was a hoax and that I was unable to withdraw my money after making a substantial investment. My business was in tatters, and I owed money.Fortunately for me, a pop-up regarding cryptocurrency recovery and cybersecurity specialists that could help me get all of my money back appeared when seeking for help on the internet. This organization, WIZARD JAMES RECOVERY, is a cyber security and cryptocurrency recovery firm that helped me get my investment money back. I sincerely appreciate their assistance and heartily endorse them to anyone in need of cryptocurrency recovery. The following is their contact information: E - M A I L: [email protected] W H A T S A P P: (+ 44 741 836 7204)

  • 11.08.25 13:05 alaincharlotte5

    RECLAIM CRYPTO LOSS: HIRE META TECH RECOVERY PRO TO RECOVER LOST ASSET BACK. The adage "all that glitters is not gold" proved a harsh reality I recently encountered. The digital realm of cryptocurrency investments and exchange platforms is not immune to deception. My pursuit of cryptocurrency investments led me to a seemingly legitimate site, complete with convincing investor testimonials touting successful withdrawals. Unbeknownst to me, it was an elaborate scheme. I invested $210,000.00 worth of cryptocurrency across multiple wallets, as directed by the scammers. Initially, the process mirrored typical modern investments, but my withdrawal requests were met with evasiveness and silence. This represented my entire life savings, accumulated through years of dedicated work, now vanished in an instant. Refusing to accept defeat, I sought assistance from METATECH-RECOVERYPRO, which facilitated a successful recovery. Their process was remarkably efficient. They meticulously traced my lost funds back to the perpetrators, and within a short timeframe, the recovered assets were returned to my bank account. Without this exceptional recovery team, which offered support when all hope seemed lost, my situation would have been dire. To initiate your recovery, I strongly advise sending a detailed email ([email protected]) of your task, outlining your specific circumstances. Ask (META TECH RECOVERY PRO) for help via: Telegram:@metatechrecoveryproteam W/S +1 (469) 692‑8049 thank you.

  • 11.08.25 13:05 alaincharlotte5

    RECLAIM CRYPTO LOSS: HIRE META TECH RECOVERY PRO TO RECOVER LOST ASSET BACK. The adage "all that glitters is not gold" proved a harsh reality I recently encountered. The digital realm of cryptocurrency investments and exchange platforms is not immune to deception. My pursuit of cryptocurrency investments led me to a seemingly legitimate site, complete with convincing investor testimonials touting successful withdrawals. Unbeknownst to me, it was an elaborate scheme. I invested $210,000.00 worth of cryptocurrency across multiple wallets, as directed by the scammers. Initially, the process mirrored typical modern investments, but my withdrawal requests were met with evasiveness and silence. This represented my entire life savings, accumulated through years of dedicated work, now vanished in an instant. Refusing to accept defeat, I sought assistance from METATECH-RECOVERYPRO, which facilitated a successful recovery. Their process was remarkably efficient. They meticulously traced my lost funds back to the perpetrators, and within a short timeframe, the recovered assets were returned to my bank account. Without this exceptional recovery team, which offered support when all hope seemed lost, my situation would have been dire. To initiate your recovery, I strongly advise sending a detailed email ([email protected]) of your task, outlining your specific circumstances. Ask (META TECH RECOVERY PRO) for help via: Telegram:@metatechrecoveryproteam W/S +1 (469) 692‑8049 thank you.

  • 11.08.25 13:05 alaincharlotte5

    RECLAIM CRYPTO LOSS: HIRE META TECH RECOVERY PRO TO RECOVER LOST ASSET BACK. The adage "all that glitters is not gold" proved a harsh reality I recently encountered. The digital realm of cryptocurrency investments and exchange platforms is not immune to deception. My pursuit of cryptocurrency investments led me to a seemingly legitimate site, complete with convincing investor testimonials touting successful withdrawals. Unbeknownst to me, it was an elaborate scheme. I invested $210,000.00 worth of cryptocurrency across multiple wallets, as directed by the scammers. Initially, the process mirrored typical modern investments, but my withdrawal requests were met with evasiveness and silence. This represented my entire life savings, accumulated through years of dedicated work, now vanished in an instant. Refusing to accept defeat, I sought assistance from METATECH-RECOVERYPRO, which facilitated a successful recovery. Their process was remarkably efficient. They meticulously traced my lost funds back to the perpetrators, and within a short timeframe, the recovered assets were returned to my bank account. Without this exceptional recovery team, which offered support when all hope seemed lost, my situation would have been dire. To initiate your recovery, I strongly advise sending a detailed email ([email protected]) of your task, outlining your specific circumstances. Ask (META TECH RECOVERY PRO) for help via: Telegram:@metatechrecoveryproteam W/S +1 (469) 692‑8049 thank you.

  • 11.08.25 17:49 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 11.08.25 17:49 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 11.08.25 17:49 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 11.08.25 19:46 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me $800,000 out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram 📱(@RoseCyberHives) and email rosecryptorecovery@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram 📱 (@RoseCyberHives) rosecryptorecovery@gmail. com

  • 11.08.25 20:06 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me $800,000 out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram 📱(@RoseCyberHives) and email rosecryptorecovery@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram 📱 (@RoseCyberHives) rosecryptorecovery@gmail. com

  • 12.08.25 02:43 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.08.25 02:43 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.08.25 02:43 michaeldavenport218

    I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR Call Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.08.25 16:48 ahmed11

    It takes a team to recover lost money from scams. Recovery professionals locate stolen money by using their expertise. The best course of action is advised by legal professionals. This helps you get your money back. Identifying warning indicators is a crucial lesson learned from frauds. Promises of large, fast, risk-free profits should be avoided. They may put pressure on you to make an investment right away. For assistance avoiding frauds in the future, get in touch with Marie. She can also help you retrieve your misplaced Bitcoin. You can reach her on Telegram at @Marie_consultancy or via email at [email protected]. Also, you can use WhatsApp at +1 7127594675.

  • 12.08.25 17:13 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.08.25 17:13 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 12.08.25 18:44 edinavida027

    CRYPTO TRACING AND RECOVERY: HIRE A LEGITIMATE RECOVERY EXPERT" CONTACT RAPID DIGITAL RECOVERY

  • 12.08.25 18:44 edinavida027

    A recruiter named Emma Zhang reached out to me on LinkedIn regarding an intriguing job opportunity that piqued my interest. She presented a position for a Crypto Security Specialist with a Coinbase-certified firm. Her profile appeared legitimate, boasting over 1200 connections and numerous endorsements from industry professionals that seemed authentic. The offer was enticing: a salary of $180,000 plus bonuses, all disbursed in Bitcoin. The recruiter promptly sent me an email from a seemingly official domain , which closely resembled Coinbase's actual email addresses. It felt like an incredible opportunity. However, there was a stipulation: to qualify for the position, I needed to confirm that I possessed at least 4 BTC. This requirement struck me as peculiar but not entirely implausible, given the nature of the role. At that moment, I had sufficient BTC to meet the demand, so I decided to proceed. The HR representative directed me to a portal that closely mimicked Coinbase's official interface. It was convincing enough that I didn’t hesitate to log in and complete the necessary steps. Unfortunately, that was where everything began to unravel. I was instructed to send a verification deposit of 1 BTC to confirm my ownership. Since everything appeared legitimate, I complied without hesitation. However, the moment I transferred the BTC, an unsettling feeling washed over me. The following day, I noticed that the remaining 3 BTC in my wallet began to vanish. My heart sank as I realized something was terribly amiss. I immediately attempted to reach out to both the HR representative and the company, but my efforts were met with silence. Upon checking the LinkedIn profiles of Emma Zhang and her team, I discovered they had mysteriously vanished. I was left with nothing no job offer, no recruiter, and no Bitcoin. I turned to the internet in search of solutions. That’s when I stumbled upon RAPID DIGITAL RECOVERY, a company specializing in Bitcoin recovery. I contacted them, and they assured me they could assist in recovering my lost funds. After a tense and arduous process, they successfully retrieved my stolen BTC. The relief I felt was indescribable, and I am profoundly grateful to RAPID DIGITAL RECOVERY for their expertise and unwavering support. CONTACT INFO BELOW Email: rapid digital recovery (@) execs. com Telegram: htt ps: // t. me/ Rapid digital recovery519 WhatSapp: +1 4 1 4 8 0 7 1 4 8 5

  • 12.08.25 19:15 vallatjosette

    Fortunately, my luck changed when I encountered Wizard Hilton Cyber Tech. Their expertise and dedication to helping victims of cryptocurrency scams are truly remarkable. They worked tirelessly to track down the scammers and recover my lost crypto coins.Email : wizardhiltoncybertech ( @ ) gmail (. ) com     OR support ( @ ) wizardhiltoncybertech (.) com

  • 12.08.25 23:07 babatunde162

    I recommend Marie when it comes to recovering lost/stolen usdt/bitcoin or any kind of cryptocurrencies' from fake investment platforms because they're well specialized in that area and you'll get your money back in full. I can boldly say this right now based on my prior deal i had with them; she was the only one who was able to recover my lost money $52,760 dollars back to my account, Only • ([email protected] and telegram:@Marie_consultancy or whatsapp +1 7127594675) she was successful in recovering my money. They are the only one who can fully restore your lost funds to your account without any deductions, I really value their work and am recommending her to you today. THANK ME LATER

  • 13.08.25 06:11 tyler231

    DO YOU REQUIRE TECHNICAL SKILLS TO SOLVE YOUR HACKING RELATED PROBLEMS? ●Hacking of all social media accounts ●Spying on cheating partner ●Retrieving of lost Cryptocurrency ●Data alteration ●Finding of lost phone ●Clearing/paying off of mortgage/loan ●Increasing of credit score ●Bitcoin mining ●Tracking of location ●Hacking of cell phone/other devices ●Block out or track down hackers Secure yourself now!!! Contact: [email protected] OR WHATSAPP+14106350697 They are the best at what they do they helped me out and i am here to testify contact them and come back testifying

  • 13.08.25 07:01 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 07:01 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 07:01 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 08:26 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 08:27 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 08:27 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 10:37 babatunde162

    Since Marie is an expert in recovering lost or stolen USDT, bitcoin, or any other cryptocurrency from fraudulent investment platforms, I suggest them for this task. You will receive your money back in full. I can confidently state this at this time because of my previous transaction with them: she was the only one who successfully returned $52,760 of my lost funds to my account. Her contact information is [email protected], and her telegram handle is @Marie_consultancy or +1 7127594675. Because they are the only ones that can completely return your lost money to your account without taking any money out, I truly appreciate what they do and am suggesting her to you today. Afterward, thank you.

  • 13.08.25 13:15 Bartlevi

    Sylvester showcased remarkable talent. He brought back over $610,000 in crypto. This felt like a massive win. His help went beyond this first success. He worked hard to get back more lost money. He stayed in touch all the time. He kept me in the loop. The whole process was very honest. There were no hidden fees or tricky ways. My money went straight to my crypto account. The transfer was easy. Sylvester Bryant is the most honest recovery agent I know. If you're in a similar spot, contact him now. You can email Sylvester at yt7cracker@gmail. com. Or reach him on WhatsApp at +1 (512) 577-7957. Don't give up hope. Sylvester can find your stolen money. He did it for me. You can get your money back. Sylvester Bryant will help you recover your funds.

  • 13.08.25 14:23 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 14:23 Morris Jason

    HIRE A REPUTABLE RECOVERY FIRM THAT HELPED ME RETRIEVE MY LOST INVESTMENT FUNDS : THE HACK ANGELS If you’re lost your Bitcoin to scammers, and you’re searching for trustworthy recovery services, getting in touch with THE HACK ANGELS is the only way to get your Bitcoin back. After losing my Bitcoin investments. I was determined to find a solution. I began by researching online and found several recovery services claiming to retrieve lost Bitcoin. However, most of these services turned out to be scams themselves. I thought I’d never see my funds again. It was then that I came across THE HACK ANGELS. A reputable Bitcoin recovery expert with a proven track record of retrieving lost Funds. I reached out to him, and he responded promptly, explaining the process of recovering my lost Bitcoin. They provide essential guidance on protecting digital assets, recognizing cyber threats and using effective security practices to prevent future incidents. Great communication and a successful outcome, when it comes to bitcoins recovery, quick contact THE HACK ANGELS RECOVERY EXPERT WhatsApp +1(520)2 0 0-2 3 2 0 Email at [email protected] Website at www.thehackangels.com If you're in London, you can even visit them in person at their office located at 45-46 Red Lion Street, London WC1R 4PF, UK. Don’t hesitate to reach out if you need help! And ready to reclaim what's rightfully yours?

  • 13.08.25 19:46 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 13.08.25 19:46 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 13.08.25 19:46 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 13.08.25 19:50 babatunde162

    When it comes to recovering lost or stolen bitcoin, USDT, or any other cryptocurrency from fraudulent investment platforms, I suggest Marie because they are highly skilled in the field and will return all of your money. Based on my previous transaction with them, I can state with confidence that she was the only one who successfully returned $52,760 of my lost funds to my account. Her email address is [email protected], and her telegram handle is @Marie_consultancy, or her WhatsApp number is +1 7127594675. I truly appreciate their work and am suggesting her to you today because they are the only ones who can completely return your missing money to your account without any deductions.

  • 14.08.25 00:20 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 14.08.25 00:20 robertalfred175

    CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 14.08.25 17:47 Joanne Ruth

    The innovative company Captain Jack Recovery was created to address the challenges associated with recovering cryptocurrency assets.  A group of seasoned professionals with backgrounds in offshore legal solutions, asset recovery, crypto intelligence, and crypto investigations founded Captain Jack Recovery.  For people and companies who have been the victims of bitcoin fraud or theft, his goal is to offer complete recovery options. Conatc address ([email protected])

  • 14.08.25 18:27 lisared

    Losing nearly 500,000 USDT to hackers was one of the most devastating experience of my life. I felt powerless as my hard-earned assets vanished into anonymous blockchain transactions until I found Agent Jasmine Lopez. Her deep expertise, tireless effort, and unwavering focus gave me hope and delivered results. She traced complex transactions, pursued every lead, and recovered a large portion of my stolen funds something I never thought possible. If you’ve lost digital assets, don’t give up. Contact her at recoveryfundprovider [AT] gmail [DOT] com Whats//App at +{44 736-644-5035}. She turned my nightmare into a recovery story, and she could do the same for you

  • 14.08.25 19:23 Joanne Ruth

    The innovative company Captain Jack Recovery was created to address the challenges associated with recovering cryptocurrency assets.  A group of seasoned professionals with backgrounds in offshore legal solutions, asset recovery, crypto intelligence, and crypto investigations founded Captain Jack Recovery.  For people and companies who have been the victims of bitcoin fraud or theft, his goal is to offer complete recovery options. Conatc address ([email protected])

  • 14.08.25 21:49 Rickbayford

    If you need to recover cryptocurrency that has been lost to wallet hackers, internet scammers, or BTC transferred to the wrong address, I suggest Wizard James Recovery, a very trustworthy recovery organization. Just by sending my case to the Expert and supplying the necessary data, Wizard James Recovery was able to help me recover my Bitcoin. I'm glad I was able to recover this much after losing even more to the fraudulent broker I initially put my belief in. Errors are unavoidable, thus we can never be careful enough. If you have lost money to cryptocurrency scams, I strongly advise getting in touch with Wizard James Recovery. Mail: ([email protected])

  • 15.08.25 03:10 KEVIN3

    HOW I RECOVER MY LOST FUNDS AFTER INVESTING Exercise extreme caution when dealing with this company. They scammed me $800,000 out of my money and even gained unauthorized access to my banking information, which severely impacted my financial well-being. I had initially approached them with the hope of improving my life through investment, but instead, I was deceived. This company should be thoroughly investigated and shut down to prevent further harm. Fortunately, I was able to recover my funds thanks to Mrs. Rose. Although I was initially hesitant and unsure about contacting her, I decided to take the risk and I’m so glad I did. You can reach her via telegram (@RoseCyberHives) and email rosecryptorecovery@gmail. com With the help of her skilled team of cybersecurity professionals and cryptocurrency experts, I was able to recover my lost funds quickly and with minimal stress, a sharp contrast to my previous experience. If you’ve been scammed or are in a similar situation, I highly recommend reaching out to her for assistance. via telegram (@RoseCyberHives) rosecryptorecovery@gmail. com

  • 15.08.25 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 04:59 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 09:13 eva117

    Recovery experts and legal experts work together to get back stolen money. They use special skills to find and retrieve assets. This teamwork improves your chances of success. Learning to spot scam warnings is key. Look out for promises of big returns with no risk. Be wary of pressure to invest fast. Contact Marie at [email protected], Telegram @Marie_consultancy, or WhatsApp +1 7127594675. She can help you avoid future scams and recover lost bitcoin

  • 15.08.25 13:27 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 13:27 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 15.08.25 20:26 caitlinfedex

    After my wife passed, I found myself lonely and searching for connection. I joined a senior living forum just to talk with others my age to share stories, maybe feel a little less isolated. That’s where I met a man who claimed to be helping retirees like me manage their savings with cryptocurrency. He spoke warmly and convincingly about a retirement care fund that guaranteed lifetime monthly payouts. He said it was specifically designed for people in my stage of life safe secure and backed by health organizations. He gained my trust over weeks of conversation. We spoke daily. He called me by name asked about my late wife, and remembered little details about my life. I believed he genuinely cared. Eventually, I transferred $200000 in USDT the majority of my retirement savings, into the wallet address he provided. That was the last time I heard from him. The forum profile vanished. His phone number stopped working. No response to emails. I was stunned. I couldn’t believe I had been tricked. I didn’t know who to turn to. I felt ashamed and foolish. At my age recovering from a financial loss like this is nearly impossible.My relative stepped in. They searched online for solutions and found TRUST GEEKS HACK EXPERT. I was skeptical at first but they insisted we give them a chance. From the moment we contacted them they treated me with respect patience and genuine concern. Their team didn’t speak in technical riddles. They explained every step. They used blockchain analysis tools to trace the movement of my funds through several addresses.It turned out the wallet was linked to a scam platform. TRUST GEEKS HACK EXPERT worked directly with the USDT issuer and coordinated with relevant platforms to freeze the funds before they were fully laundered.In just two weeks they managed to recover $155000 in USDT.The money helped yes but it was more than that. I no longer felt powerless. I had been taken advantage of but I wasn’t alone in the fight. Thanks to TRUST GEEKS HACK EXPERT I got back not only a large part of my savings but also peace of mind a sense of dignity and hope for the future. If you ever find yourself in such situation I encourage you to contact TRUST GEEKS HACK EXPERT Email.trustgeekshackexpert{At}fastservice{Dot}com — Telegram:: Trustgeekshackexpert & W h a t's A p p  +1 7 1 9 4 9 2 2 6 9 3

  • 15.08.25 20:27 caitlinfedex

    After my wife passed, I found myself lonely and searching for connection. I joined a senior living forum just to talk with others my age to share stories, maybe feel a little less isolated. That’s where I met a man who claimed to be helping retirees like me manage their savings with cryptocurrency. He spoke warmly and convincingly about a retirement care fund that guaranteed lifetime monthly payouts. He said it was specifically designed for people in my stage of life safe secure and backed by health organizations. He gained my trust over weeks of conversation. We spoke daily. He called me by name asked about my late wife, and remembered little details about my life. I believed he genuinely cared. Eventually, I transferred $200000 in USDT the majority of my retirement savings, into the wallet address he provided. That was the last time I heard from him. The forum profile vanished. His phone number stopped working. No response to emails. I was stunned. I couldn’t believe I had been tricked. I didn’t know who to turn to. I felt ashamed and foolish. At my age recovering from a financial loss like this is nearly impossible.My relative stepped in. They searched online for solutions and found TRUST GEEKS HACK EXPERT. I was skeptical at first but they insisted we give them a chance. From the moment we contacted them they treated me with respect patience and genuine concern. Their team didn’t speak in technical riddles. They explained every step. They used blockchain analysis tools to trace the movement of my funds through several addresses.It turned out the wallet was linked to a scam platform. TRUST GEEKS HACK EXPERT worked directly with the USDT issuer and coordinated with relevant platforms to freeze the funds before they were fully laundered.In just two weeks they managed to recover $155000 in USDT.The money helped yes but it was more than that. I no longer felt powerless. I had been taken advantage of but I wasn’t alone in the fight. Thanks to TRUST GEEKS HACK EXPERT I got back not only a large part of my savings but also peace of mind a sense of dignity and hope for the future. If you ever find yourself in such situation I encourage you to contact TRUST GEEKS HACK EXPERT Email.trustgeekshackexpert{At}fastservice{Dot}com — Telegram:: Trustgeekshackexpert & W h a t's A p p  +1 7 1 9 4 9 2 2 6 9 3

  • 15.08.25 21:04 vallatjosette

    My Wild Ride: How a Wizard Hilton Cyber Tech Helped Me Recover Lost Bitcoin of 103,000 It all started when I had the misfortune of misplacing the digital wallet containing my substantial cryptocurrency holdings - a staggering 103,000 bitcoins. I had amassed this sizable fortune over the years through strategic investments and careful trading, but in a moment of carelessness, the private keys to my digital wallet had vanished. Panic set in as I grappled with the gravity of my situation - those bitcoins, worth millions at the time, were effectively lost to the ether, inaccessible and seemingly unrecoverable. Just when I had resigned myself to the devastating financial blow, a chance encounter with a self-proclaimed "digital wizard" Wizard Hilton Cyber Tech presented a glimmer of hope. This enigmatic individual, who operated in the shadowy corners of the crypto underworld, claimed to possess the arcane knowledge and skills necessary to retrieve my lost digital wealth. Intrigued and desperate, I put my skepticism aside and entrusted this mysterious wizard with the task. What followed was a wild, suspenseful journey into the depths of blockchain technology and cryptography, as the wizard navigated complex algorithms, exploited vulnerabilities, and pieced together the puzzle of my missing private keys. After days of intense effort, Wizard Hilton Cyber Tech emerged triumphant, having successfully recovered the entirety of my 103,000 bitcoin stash. The sheer relief and elation I felt in that moment was indescribable, as I regained access to my small digital fortune and was able to secure my financial future. This harrowing experience taught me the invaluable lesson of never taking the security of my cryptocurrency holdings for granted, and the importance of always maintaining the utmost diligence when it comes to protecting one's digital wealth. Email: wizardhiltoncybertech ( @ ) gmail (. ) com     OR support ( @ ) wizardhiltoncybertech (.) com WhatsApp number  +18737715701 Thanks.

  • 16.08.25 06:53 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 16.08.25 06:53 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 16.08.25 06:53 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 16.08.25 07:01 lisared

    Losing nearly 500,000 USDT to hackers was one of the most devastating experience of my life. I felt powerless as my hard-earned assets vanished into anonymous blockchain transactions until I found Agent Jasmine Lopez. Her deep expertise, tireless effort, and unwavering focus gave me hope and delivered results. She traced complex transactions, pursued every lead, and recovered a large portion of my stolen funds something I never thought possible. If you’ve lost digital assets, don’t give up. Contact Recoveryfundprovider@gmail. com WhatsApp at +{44 736-644-5035}. She turned my nightmare into a recovery story, and she could do the same for you.

  • 16.08.25 17:05 mark

    I see a lot of recommendations online and it’s already obvious there are bad eggs online who will only add to your mystery. I can only recommend one and you can reach them via mail on (zattechrecovery @ gmail c0 m ) if you need help on recovering what you lost to scammers.

  • 17.08.25 03:33 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.08.25 03:33 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.08.25 05:22 aveasley4

    I’m a California resident who was contacted by someone named “Lin” on WhatsApp. Our conversation eventually turned to cryptocurrency investments. Lin introduced me to a trading platform called CBOTBIT (https://cbotbit.com), walked me through setting up an account, and guided me on transferring funds. Under Lin’s direction, I began trading and was led to believe that my account was growing significantly. However, when I attempted to withdraw my funds, I was unable to do so — and shortly after, the website went offline entirely. I had invested over $340,000 in Bitcoin, planning to use it for my retirement. The experience was devastating, and I was at a loss for what to do next. Eventually, I came across [email protected] and decided to reach out. I followed their instructions, and within a few days, I was able to recover my original investment. I'm incredibly relieved and grateful for their assistance, and I recommend contacting them if you’re in a similar situation and need help recovering lost funds.

  • 17.08.25 17:49 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.08.25 17:49 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 17.08.25 17:49 marcushenderson624

    Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 18.08.25 21:10 Connorhelms

    I can honestly say losing my bitcoin savings was one of the darkest moments of my life. I had put everything I could into that investment, believing it would secure my future, only to wake up one day and realize the platform had vanished with all my funds. The feeling was indescribable, fear, shame, and helplessness all at once. I thought I’d never see that money again. Then I came across Alpha Spy Nest. At first, I was hesitant, I had already been deceived once, and the thought of trusting anyone else terrified me. But something about their professionalism and the way they patiently explained things gave me a little hope. For the first time in weeks, I felt like I wasn’t completely alone in the fight. They listened to my story without judgment, broke down the technical side in a way I could understand, and reassured me that there was still a chance. Watching them work was incredible. They traced every transaction, uncovered where my bitcoin had been moved, and kept me updated through every stage. It felt like they cared as much about my recovery as I did. The day they told me they had successfully recovered my lost savings, I actually cried. Seeing those funds back in my wallet felt unreal, it was like a huge weight had been lifted off my shoulders. Beyond just recovering my bitcoin, Alpha Spy Nest gave me back my peace of mind and restored a trust I thought I had lost forever. I’m forever grateful to them. If I hadn’t found Alpha Spy Nest, I’d still be stuck in regret and despair. Instead, I now share my story as proof that recovery is possible. You can also reach out to them via: Email: [email protected], WhatsApp: +15132924878‬, Telegram: https://t.me/Alphaspynest, Website: www.alphaspynest.org

  • 18.08.25 22:17 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

  • 18.08.25 22:17 wendytaylor015

    My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1

Для участия в Чате вам необходим бесплатный аккаунт pro-blockchain.com Войти Регистрация
Есть вопросы?
С вами на связи 24/7
Help Icon