Этот сайт использует файлы cookies. Продолжая просмотр страниц сайта, вы соглашаетесь с использованием файлов cookies. Если вам нужна дополнительная информация, пожалуйста, посетите страницу Политика файлов Cookie

Прямой эфир

Русский

English

Войти / Регистрация

Cryptocurrencies: 9380 / Markets: 114950

Market Cap: $ 3 644 332 247 038 / 24h Vol: $ 186 288 672 461 / BTC Dominance: 59.263178941211%

Н Новости

[Перевод] Оценка систем LLM: основные метрики, бенчмарки и лучшие практики

Оценка LLM-систем вручную — это утомительное, времязатратное и порой раздражающее занятие. Поэтому, если вам приходилось многократно прогонять набор промптов, чтобы вручную проверять каждый вывод LLM, вас обрадует, что эта статья научит вас всему необходимому для правильной оценки LLM и обеспечения долгосрочной эффективности вас и вашего приложения на основе LLM.

Оценка LLM относится к процессу обеспечения соответствия выходных данных LLM человеческим ожиданиям, которые могут варьироваться от этических и безопасных соображений до более практических критериев, таких как правильность и релевантность выходных данных LLM. С инженерной точки зрения эти выходные данные LLM часто можно найти в форме тестовых кейсов, в то время как критерии оценки формализуются в виде метрик оценки LLM.

На повестке дня:

В чем разница между оценкой LLM и оценкой системы LLM, а также их преимущества
Офлайн-оценки, что такое бенчмарки системы LLM, как создавать наборы данных для оценки и выбирать правильные метрики оценки LLM, а также распространенные ошибки
Оценки в реальном времени и их польза для улучшения наборов данных для офлайн-оценок
Реальные примеры использования систем LLM и как их оценивать, включая chatbotQA и Text-SQL

Давайте начнем.

Оценка LLM vs оценка системы LLM

Давайте разберемся: в то время как LLM (большая языковая модель) относится конкретно к модели (например, GPT-4), обученной понимать и генерировать человеческий язык. LLM-система, в свою очередь, представляет собой полную инфраструктуру, которая включает не только саму LLM, но и дополнительные компоненты, такие как вызовы функций (для агентов), системы поиска (в RAG), кэширование ответов и т.д., что делает LLM полезными для реальных приложений, таких как чат-боты поддержки клиентов, автономные агенты продаж и генераторы Text-to-SQL.

Однако важно отметить, что система LLM иногда может состоять из одной LLM, как в случае с ChatGPT. Вот пример системы LLM на основе RAG, которая выполняет задачу Text-to-SQL:

Гипотетическая архитектура Text-SQL

Основной целью Text-to-SQL является генерация правильного и эффективного SQL для заданного пользовательского запроса, поэтому пользовательский запрос обычно сначала используется для извлечения соответствующих таблиц в схеме базы данных через конвейер извлечения, прежде чем использовать его в качестве контекста для генерации правильного SQL через конвейер генерации SQL. Вместе они составляют систему LLM (основанную на RAG).

(Примечание: технически вам не нужно выполнять извлечение перед генерацией, но даже для схемы базы данных среднего размера лучше выполнить извлечение, чтобы LLM меньше галлюцинировала).

Поэтому оценка системы LLM не так проста, как оценка самого LLM. Хотя и LLM, и системы LLM получают и генерируют текстовые выходные данные, тот факт, что в системе LLM может быть несколько компонентов, работающих совместно, означает, что вам следует применять метрики оценки LLM более детально для оценки различных частей системы LLM для максимальной видимости того, где что-то идет не правильно (или правильно).

Например, вы можете применить метрику контекстной полноты для оценки конвейера извлечения в примере Text-to-SQL выше, чтобы оценить, может ли он извлечь все необходимые таблицы, необходимые для ответа на конкретный запрос пользователя.

Метрику контекстной полноты зачастую можно использовать для оценки конвейеров извлечения

Аналогичным образом вы можете применить пользовательскую метрику корректности SQL, реализованную через G-Eval, для оценки того, генерирует ли конвейер генерации правильный SQL на основе извлеченных таблиц данных top-K.

Пользовательская метрика корректности SQL может использоваться для оценки выходных данных SQL

Подводя итог, можно сказать, что система LLM состоит из нескольких компонентов, которые помогают сделать LLM более эффективным в выполнении своей задачи, как показано в примере Text-to-SQL, и ее сложнее оценить из-за ее сложной архитектуры.

В следующем разделе мы рассмотрим, как можно выполнить оценку системы LLM в процессе разработки (офлайн-оценки), включая способы, с помощью которых мы можем быстро создать большое количество тестовых кейсов для модульного тестирования нашей системы LLM, а также как выбрать правильные метрики оценки LLM для определенных компонентов.

Офлайн-оценки: тестовые кейсы, наборы данных для оценки, метрики LLM и бенчмарки

Офлайн-оценки относятся к оценке или тестированию вашей системы LLM в локальной среде разработки. Представьте себе выполнение оценок в вашей локальной среде разработки, которая может быть чем угодно: от скрипта Python, Colab/Jupyter Notebooks до конвейеров CI/CD в Github Actions перед развертыванием в продакшн. Это позволяет количественно улучшить архитектуру вашей LLM-системы или даже подобрать идеальный набор гиперпараметров (например, используемую LLM, шаблоны промптов, модель эмбеддингов и т.д. для каждого компонента).

Бенчмаркинг систем LLM

Некоторые инженеры называют этот процесс "бенчмаркингом" LLM-систем, и это абсолютно верно. Бенчмаркинг LLM-систем — это процесс количественного измерения их производительности на основе набора пользовательских критериев, используя метрики оценки LLM и стандартизированный набор данных для оценки. Именно так проводится оффлайн-оценка. Исходя из примера Text-to-SQL в предыдущем разделе, метрики, используемые для сравнительного анализа нашей системы LLM, будут контекстной полнотой и пользовательской метрикой корректности SQL. Эти метрики оценивают, насколько хорошо LLM-система Text-to-SQL справляется с набором пользовательских запросов и выводами SQL, которые мы называем тестовыми кейсами.

Что касается терминологии, вот что вам нужно знать:

Бенчмарки состоят из (обычно одного) набора данных для оценки и метрик оценки LLM.
Набор данных для оценки состоит из тестовых кейсов, к которым будут применяться метрики оценки LLM.
Для одного бенчмарка может быть несколько метрик оценки LLM, каждая из которых оценивает различные части системы LLM для каждого тестового кейса.

Вы можете запускать эти бенчмарки системы LLM в локальной среде разработки каждый раз, когда вносите изменения в свою систему LLM (например, переключаетесь на другую архитектуру извлечения или поставщика LLM), чтобы обнаружить любые критические изменения в производительности, или вы можете даже интегрировать эти бенчмарки в конвейер CI/CD для регрессионного тестирования системы LLM.

Прежде чем перейти к следующему разделу, вот спойлер о том, как вы можете настроить все в коде с помощью DeepEval, фреймворка оценки LLM с открытым исходным кодом:

pip install deepeval

from deepeval.dataset import EvaluationDataset
from deepeval.metrics import ContextualRecallMetric

metric = ContextualRecallMetric(threshold=
0.5
)

dataset = EvaluateionDataset(test_cases=[...])
dataset.evaluate(metrics=[metric)

И для тех, кто хочет начать оценку немедленно, ознакомьтесь с кратким руководством DeepEval.

Наборы данных для оценки и тестовые кейсы

Набор данных для оценки — это набор тестовых кейсов.

Тестовый кейс LLM

Тестовый кейс — это единица данных, содержащая всю необходимую информацию для модульного тестирования вашей LLM-системы. Вот основные параметры, которые входят в тестовый кейс для LLM:

Входные данные — входные данные для вашей системы LLM.
Фактические выходные данные — текстовый ответ, сгенерированный вашей системой LLM.
Ожидаемые выходные данные — идеальный ответ для заданных входных данных для вашей системы LLM.
Контекст извлечения — текстовые фрагменты/документы, извлеченные в конвейере RAG/системе извлечения.
Контекст — идеальные текстовые фрагменты/документы, из которых выводятся ожидаемые выходные данные.

Вот в чем подвох: разные метрики требуют заполнения разных параметров в тестовом кейсе LLM. Например, метрика релевантности ответа без эталона (reference-less answer relevancy metric, о которой вы узнаете позже) требует только входных данных и фактического вывода. В то же время метрика релевантности ответа с эталоном (reference-based answer relevancy metric) потребует входных данных, фактического вывода и ожидаемого результата.

Другой пример вы можете найти в исходной метрике контекстной полноты, использованной ранее в примере Text-to-SQL. Поскольку контекстная полнота оценивает, способна ли ваша система извлечения извлекать все соответствующие фрагменты текста для ответа на определенный запрос, вам понадобятся входные данные (т. е. запрос) и контекст извлечения (т. е. соответствующие фрагменты текста, которые были извлечены).

Вот несколько правил, которым необходимо следовать при подготовке набора данных оценки:

Начните с 50–100 тестовых кейсов. Вы можете либо сформировать их с нуля, либо использовать данные из продакшена (если ваша LLM-система уже развернута). На практике это количество оптимально: оно достаточно для выявления слабых мест в вашей системе и не слишком велико для начальной работы.
Подбирайте тестовые кейсы которые наиболее вероятно могут не соответствовать критериям, выбранным для оценки вашей LLM-системы.
Фактические выходные данные не обязательно вычислять заранее, особенно если вы планируете тестировать вашу систему после каждого изменения.
Заполните тестовые кейсы ожидаемыми выходными данными. Как было объяснено в разделе о метриках LLM, метрики с использованием эталонных данных (reference-based metrics) обеспечивают гораздо более точные результаты. Поэтому важно заранее определить ожидаемые выходные данные.

Подготовить тестовые кейсы с ожидаемыми выходными данными - это определенно непростая задача, особенно для 50–100 тестовых кейсов. В настоящее время существует два основных способа создания наборов данных для оценки для бенчмаркинга вашей системы LLM:

Человеческая аннотация — это просто: наймите кого-нибудь, чтобы он подготовил 100 тестовых кейсов. Можно также использовать данные из производственной среды, если ваша система уже развернута.
Генерация синтетических данных. Этот метод немного сложнее: использовать LLM для генерации 100 тестовых кейсов.

Генерация синтетических данных с помощью LLM включает предоставление LLM контекста (того же контекста в тестовом кейсе LLM, как описано выше) и генерацию соответствующих входных данных и ожидаемых выходных данных на основе заданного контекста.

Например, в случае использования Text-to-SQL хороший генератор синтетических данных будет:

Случайным образом (для разнообразия набора данных) выбирать разные таблицы в вашей базе данных и использовать их как контекст тестового кейса.
Генерировать пользовательский запрос в качестве входных данных для вашего тестового кейса.
Генерировать ожидаемые выходные данные на основе контекста и входных данных вашего тестового кейса.
Повторять эти шаги до тех пор, пока не будет создан разумный объем данных для оценки.

Однако необходимо развивать ввод тестовых кейсов, чтобы сделать его более реалистичным и неотличимым от данных, подготовленных вручную. Этот процесс называется эволюцией, и он был изначально представлен в статье Evol-Instruct от Microsoft.

Вот быстрый способ сгенерировать синтетические данные с использованием LLM через DeepEval:

from deepeval.dataset import EvaluationDataset

dataset = EvaluationDataset()
dataset.generate_goldens_from_docs(
    document_paths=['example.pdf'],
    include_expected_output=True,
    max_goldens_per_document=
5

)

Обратите внимание, что в DeepEval мы генерируем «голдены», которые по сути являются тестовыми кейсами, но без фактических выходных данных. Это означает, что во время оценки вам нужно будет запустить вашу систему LLM для генерации фактических выходных данных для каждого голдена, чтобы сделать их полноценными тестовыми кейсами.

Метрики LLM

Метрики оценки LLM используются для автоматизации процесса оценки систем LLM. Конечно, вы можете поручить оценку систем LLM людям, но тогда вы вернетесь к неэффективной проверке выходных данных LLM. Поэтому вопрос состоит в том, как создать свою собственную метрику оценки LLM или какие метрики оценки LLM следует использовать для вашей системы LLM?

В одной из моих предыдущих статей я подробно рассказал обо всем, что вам нужно знать о метриках оценки LLM, но вкратце повторю - традиционные метрики, такие как BERT, ROUGE и n-gram, не работают, потому что выходные данные LLM слишком сложны для оценки статистическими методами или моделями, не основанных на LLM. Решение — использовать подход LLM-as-a-judge («LLM в роли судьи»).

LLM-as-a-judge предполагает использование LLM для оценки тестовых кейсов. Возможно, вы сначала отнесетесь к этому подходу скептически, но исследования показали, что использование LLM для оценки других LLM — это максимально близкий к человеческому подход корреляции.

Более высокая корреляция Спирмена и Кендалла-Тау отражает более высокую согласованность с человеческим суждением.

Более того, такие подходы, как G-Eval, использование промптов для уменьшения объема текста, который нужно оценить в тестовом кейсе, или применение метода QAG (Question Answer Generation) для математически более точного вычисления оценки метрики для каждого тестового кейса, помогают LLM генерировать результаты, гораздо лучше соответствующие ожиданиям человека. (Я настоятельно рекомендую прочитать эту статью, если у вас есть какие-либо вопросы или вы хотите узнать больше о том, как использовать LLM для оценки других LLM.)

Ниже приведен список наиболее часто используемых метрик для оценки LLM-систем, каждая из которых оценивает различные аспекты системы:

Корректность
Релевантность ответа
Достоверность
Сходство
Связность
Контекстная полнота
Контекстная релевантность
Контекстная точность
Предвзятость
Токсичность

(DeepEval поддерживает каждую из этих метрик «из коробки», и вы можете ознакомиться с документацией, чтобы узнать больше о каждой метрике и ситуациях, в которых ее следует использовать.)

Например, вот как можно реализовать пользовательскую метрику корректности с подходом «LLM в роли судьи» на практике:

from deepeval.metrics import GEval
from deepeval.test_case import LLMTestCaseParams

correctness_metric = GEval(
    name="Correctness",
    criteria="Determine whether the actual output is factually correct based on information in the expected output.",
    evaluation_params=[
      LLMTestCaseParams.ACTUAL_OUTPUT, 
      LLMTestCaseParams.EXPECTED_OUTPUT
    ]
)

Когда вы проводите оценки на стадии разработки, следует стремиться использовать метрики на основе эталонов (reference-based metrics). Эталонные метрики сравнивают результат модели с заранее подготовленным «золотым» стандартом или идеальным ожидаемым выводом в тестовом кейсе.

Метрика релевантности ответа: безэталонная и эталонная

Хотя вы можете вычислить оценку релевантности ответа, используя LLM для оценки того, насколько релевантен вывод модели по отношению к входным данным, лучше использовать эталонные выводы и сравнивать их с выводом LLM. Это связано с тем, что метрики без эталонов обычно опираются на более «свободные» критерии, из-за чего оценки метрик, вычисленных LLM, имеют тенденцию к большей вариативности для одного и того же набора тестовых кейсов по сравнению с эталонными метриками. Иными словами, эталонные метрики более надежны.

Да, на создание и подготовку эталонных тестовых наборов уходит больше времени и усилий, но, имея опыт поддержки сотен сценариев оценки LLM через DeepEval, я могу с уверенностью сказать, что это позволяет значительно точнее тестировать LLM-системы.

В следующем разделе я покажу вам, как можно полуавтоматизировать этот процесс, сгенерировав синтетические наборы данных/тестовые кейсы с использованием LLM.

Бенчмарки

Бенчмарки состоят из наборов данных для оценки и метрик LLM, и в отличие от типичных бенчмарков для LLM, все бенчмарки систем LLM являются кастомными. Это связано с тем, что:

Не существует стандартных метрик оценки LLM для систем LLM (даже для данного варианта использования, такого как Text-to-SQL), поскольку метрики зависят от точной архитектуры системы LLM, и не существует стандартной архитектуры для любого отдельного варианта использования.
Не существует стандартного набора тестовых кейсов/набора данных для оценки для конкретного варианта использования. Например, вам понадобятся два разных набора данных оценки для двух разных схем базы данных для варианта использования Text-to-SQL, поскольку пользовательские запросы и golden выходные данные SQL в тестовых кейсах будут разными.

И пока я не забыл, вот как вы можете протестировать свою систему LLM в коде, используя метрику релевантности ответа:

from deepeval.dataset import EvaluationDataset
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric

test_case = LLMTestCase(
  input="Why did the chicken cross the road?", 
  actual_output="Quite frankly, I don't want to know..."
)
metric = AnswerRelevancyMetric()

dataset = EvaluationDataset(test_cases=[test_case])
dataset.evaluate(metrics=[metric])

Типичные ошибки в оффлайн-оценке

Самая большая ошибка, которую можно совершить, — это предположить, что ваш набор данных для бенчмарка «достаточно хорош», чтобы его никогда не улучшать. Улучшение набора данных для бенчмарка имеет решающее значение, поскольку для того, чтобы ваш бенчмарк имел смысл, он всегда должен отражать самые последние уязвимости в вашем приложении LLM.

В следующем разделе я покажу вам, как вы можете улучшить свой набор данных для оценки бенчмарка, когда вы переводите свою систему LLM в производство.

Оценки в реальном времени

(Если ваша система LLM ещё не в продакшене, этот раздел может быть вам не актуален.)

Офлайн-оценка отлично подходит для итераций, но она ограничена полнотой вашего набора данных для тестирования. Хотя вы можете привлекать аннотаторов или использовать инструменты синтеза данных для создания новых тестовых кейсов на основе данных из вашей базы знаний, важно также включать данные из продакшена в ваши бенчмарки, чтобы симулировать взаимодействие реальных пользователей с вашим приложением LLM.

Данные из продакшена включают входные данные и ответы, которые ваша система LLM получает и генерирует в реальной рабочей среде. Но вот в чём проблема: у вас, скорее всего, не будет времени и ресурсов, чтобы проверить все ответы, сгенерированные системой LLM в продакшене, и решить, какие из них добавить в набор данных для оценки.

Представьте, что ваше приложение LLM генерирует 50 000 ответов в месяц. Скорее всего, вы не сможете просмотреть их все, не потеряв рассудок. Вы можете случайным образом выбрать 10% ответов для проверки вручную, но даже это останется крайне трудозатратной задачей. Именно поэтому оценка в реальном времени столь ценна.

Оценки в реальном времени позволяют вам легко определить ответы, которые, скорее всего, будут неудовлетворительными. Вместо случайного отбора 10% ответов, сгенерированных в продакшене, надёжная инфраструктура оценки в реальном времени позволяет фильтровать ответы, которые не соответствуют определённым критериям. Это помогает человеческим экспертам более эффективно улучшать бенчмарки вашей системы LLM, используя данные из продакшена.

Метрики LLM в продакшене

В предыдущем разделе о метриках LLM мы подчёркивали, что на этапе разработки следует использовать метрики, основанные на эталонных данных (reference-based metrics), так как они делают результаты более надёжными благодаря строгим критериям оценки.

Однако метрики, основанные на эталонных данных, невозможно применять в продакшене, так как у вас нет ожидаемых результатов для каждого сгенерированного ответа. Поэтому в продакшене применяются метрики, не основанные на эталонных данных. Ниже приведён пример метрики релевантности ответа без эталонных данных (хотя ранее мы вводили её как метрику на основе эталонов):

Метрика релевантности ответа без использования эталонов

Суть подхода в том, чтобы выбрать одну или несколько метрик, не зависящих от эталонов, использовать их для оценки каждого нового ответа системы LLM, а затем на основе этих оценок определять, какие ответы следует добавить для расширения существующего набора данных для оценки.

Примеры использования

В заключение я подготовил два простых примера оффлайн-оценки LLM для новых вариантов использования системы LLM. Для каждого примера мы выполним следующие шаги:

Генерация синтетических тестовых кейсов для вашего бенчмарка
Выбор соответствующих метрик оценки LLM
Запуск бенчмарка

Чат-бот QA

Чат-бот QA (вопрос-ответ) — это по сути поисковая система на базе чата, но в отличие от ChatGPT он не является полностью разговорным. Обычно используется для быстрого поиска информации в базе знаний. Такой чат-бот применяет подход RAG: сначала он извлекает релевантную информацию с помощью векторного поиска по базе данных, содержащей проанализированную и индексированную информацию из базы знаний, а затем передаёт эту информацию в LLM для генерации ответа на пользовательский запрос:

Гипотетическая архитектура чат-бота QA

(Примечание: эта диаграмма и схема Text-to-SQL выглядят почти одинаково, поскольку оба варианта использования представляют собой упрощенную систему LLM на основе RAG)

Шаг 1: Предполагая, что у нас уже есть несколько PDF-документов, представляющих нашу базу знаний, под названием knowledge_1.pdf, knowledge_2.pdf и т. д., мы можем сгенерировать синтетические тестовые кейсы из этих документов для нашего чат-бота QA:

from deepeval.synthesizer import Synthesizer

synthesizer = Synthesizer()
goldens = synthesizer.generate_goldens_from_docs(
    document_paths=[
      'knowledge_1.pdf', 
      'knowledge_1.pdf',
      'knowledge_1.pdf',
      'knowledge_1.pdf',
      'knowledge_1.pdf',
      'knowledge_1.pdf'

    ],
    max_goldens_per_document=
15
,
    include_expected_output=True

Шаг 2: Теперь, когда у нас есть некоторые сгенерированные синтетические данные, мы можем определить наши метрики. Для этого примера мы будем использовать только две метрики: корректность и контекстную полноту.

from deepeval.metrics import GEval, ContextualRecallMetric
from deepeval.test_case import LLMTestCaseParams

contextual_recall = ContextualRecallMetric(threshold=
0.6
)
correctness = GEval(
    name="Correctness",
    criteria="Determine whether the actual output is factually correct based on the expected output.",
    evaluation_params=[
      LLMTestCaseParams.ACTUAL_OUTPUT, 
      LLMTestCaseParams.EXPECTED_OUTPUT
    ],
    threshold=
0.6

)

(Ссылка на документацию по метрикам здесь)

Метрика корректности G-Eval сравнивает фактическую правильность фактических выходных данных с ожидаемыми выходными данными, в то время как контекстная полнота оценивает, содержит ли контекст поиска всю информацию, необходимую для генерации ожидаемых выходных данных для заданных входных данных. Также обратите внимание, что мы установили порог прохождения для каждой метрики на 0,6; это означает, что тестовый кейс будет пройден, если обе метрики наберут оценку выше или равную 0,6.

(PS: Прокрутите вверх и снова посмотрите диаграмму Chatbot QA, чтобы понять, какую часть системы LLM оценивает каждая метрика.)

Шаг 3: Имея готовый набор данных оценки и метрики, мы можем запустить бенчмарк. Мы не собираемся внедрять систему чат-бота QA, но предположим, что у нас есть гипотетическая функция run_chatbot_qa(), которая возвращает сгенерированный ответ и извлеченные узлы, используемые для генерации ответа:

llm_output, retrieved_nodes = run_chatbot_qa("A user query")

Мы можем запустить бенчмарк, сначала преобразовав голдены, сгенерированные на шаге 1, в тестовые кейсы, готовые к оценке:

from deepeval.test_case import LLMTestCase
from deepeval.dataset import EvaluationDataset
...

test_cases = []
for golden in goldens:
  query = golden.input
  llm_output, retrieved_nodes = run_chatbot_qa(query)
  test_case = LLMTestCase(
    input=query,
    actual_output=llm_output,
    retrieval_context=retrieved_nodes,
    expected_output=golden.expected_output   
  )
  test_cases.append(test_case)

dataset = EvaluationDataset(test_cases=test_cases)

(Ссылка на документацию по датасету здесь)

И наконец, запустите свой бенчмарк, применяя к нашему набору данных две выбранные метрики:

...

dataset.evaluate(metrics=[correctness, contextual_recall])

Это было не так уж и сложно, верно?

Text-SQL

Text-SQL — это сценарий использования, в котором система LLM генерирует соответствующий SQL-запрос для ответа на аналитический вопрос о данных, например, «Сколько пользователей зарегистрировалось на прошлой неделе»?

Мы собираемся повторно использовать ту же диаграмму архитектуры Text-SQL, что и в начале этой статьи:

Снова гипотетическая архитектура Text-SQL

Шаг 1: На этом этапе генерации данных мы будем создавать синтетические эталонные данные (goldens) с использованием таблиц данных в схеме базы данных.

from deepeval.synthesizer import Synthesizer, UseCase

table1 = """CREATE TABLE Students (
    StudentID INT PRIMARY KEY,
    FirstName VARCHAR(50),
    LastName VARCHAR(50),
    Email VARCHAR(100) UNIQUE,
    DateOfBirth DATE,
    Gender CHAR(1),
    Address VARCHAR(200),
    PhoneNumber VARCHAR(15)
);"""

table2 = """CREATE TABLE Courses (
    CourseID INT PRIMARY KEY,
    CourseName VARCHAR(100),
    TeacherID INT,
    Credits INT,
    DepartmentID INT,
    FOREIGN KEY (TeacherID) REFERENCES Teachers(TeacherID),
    FOREIGN KEY (DepartmentID) REFERENCES Departments(DepartmentID)
);"""

table3 = """CREATE TABLE Enrollments (
    EnrollmentID INT PRIMARY KEY,
    StudentID INT,
    CourseID INT,
    EnrollmentDate DATE,
    Grade CHAR(2),
    FOREIGN KEY (StudentID) REFERENCES Students(StudentID),
    FOREIGN KEY (CourseID) REFERENCES Courses(CourseID)
);"""

table4 = """CREATE TABLE Teachers (
    TeacherID INT PRIMARY KEY,
    FirstName VARCHAR(50),
    LastName VARCHAR(50),
    Email VARCHAR(100) UNIQUE,
    DepartmentID INT,
    FOREIGN KEY (DepartmentID) REFERENCES Departments(DepartmentID)
);"""

synthesizer = Synthesizer()
goldens = synthesizer.generate_goldens(
    contexts=contexts,
    use_case=UseCase.TEXT2SQL,
    max_goldens_per_context=
15

)

Шаг 2: Мы будем использовать пользовательскую метрику корректности SQL и метрику контекстной полноты:

from deepeval.metrics import ContextualRecallMetric

contextual_recall = ContextualRecallMetric(threshold=
0.5
)
sql_correctness = GEval(
    name="SQL Correctness",
    evaluation_steps=[
        "Check the `actual output` for SQL syntax errors.",
        "Verify that the `actual output` SQL is compatible with the schema in the `retrieval context`, ensuring correct usage of tables and columns.",
        "Execute the `actual output` SQL and ensure the results precisely match the `expected output`.",
        "Confirm that the `actual output` SQL is indeed relevant to the `input` user query.",
    ],
    evaluation_params=[
        LLMTestCaseParams.INPUT,
        LLMTestCaseParams.ACTUAL_OUTPUT,
        LLMTestCaseParams.EXPECTED_OUTPUT,
        LLMTestCaseParams.RETRIEVAL_CONTEXT
    ],
)

(Ссылка на документацию по метрикам здесь)

Подобно метрикам, используемым в чат-боте QA, метрика корректности SQL и контекстной полноты оценивает генератор запросов и конвейер извлечения соответственно.

Шаг 3: Предположим, что у нас есть еще одна гипотетическая функция run_text_2_sql(), которая возвращает сгенерированный SQL-запрос и извлеченную таблицу, использованные для его генерации:

sql_output, retrieved_tables = run_text_2_sql("How many users signed up last week?")

Мы можем запустить бенчмарк, сначала преобразовав голдены, сгенерированные в шаге 1, в тестовые кейсы, готовые к оценке:

from deepeval.test_case import LLMTestCase
from deepeval.dataset import EvaluationDataset
...

test_cases = []
for golden in goldens:
  query = golden.input
  sql_output, retrieved_tables = run_text_2_sql(query)
  test_case = LLMTestCase(
    input=query,
    actual_output=sql_output,
    retrieval_context=retrieved_tables,
    expected_output=golden.expected_output   
  )
  test_cases.append(test_case)

dataset = EvaluationDataset(test_cases=test_cases)

(Ссылка на документацию по датасету здесь)

И наконец, запустите свой бенчмарк, применяя к нашему набору данных две выбранные метрики:

...

dataset.evaluate(metrics=[sql_correctness, contextual_recall])

Заключение

Поздравляем вас с тем, что вы дошли до этого момента! В этой статье мы узнали разницу между LLM и системами LLM и как мы можем оценивать их разными методами.

Мы познакомились с методами оффлайн-оценки, включая концепцию бенчмаркинга с использованием пользовательских наборов данных и метрик для регрессионного тестирования систем LLM, и рассмотрели, как инструмент DeepEval поддерживает весь цикл оценки. Мы также подчеркнули важность регулярного обновления наборов данных для бенчмаркинга и рассмотрели, как оценки в реальном времени в продакшене могут способствовать этому процессу. И наконец, мы подробно разобрали два распространённых сценария применения систем LLM — чатбот для вопросов и ответов (Chatbot QA) и генерацию SQL-запросов (Text-to-SQL) — и изучили подходы к их оценке.

Источник

Теги

Категория

Новости

Дата

22 янв. 2025 г.

09.10.25 08:23 pHqghUme

(select(0)from(select(sleep(15)))v)/*'+(select(0)from(select(sleep(15)))v)+'"+(select(0)from(select(sleep(15)))v)+"*/
09.10.25 08:24 pHqghUme

is it ok if I upload an image?
09.10.25 08:24 pHqghUme

e
09.10.25 08:24 pHqghUme

can I ask you a question please?-1 waitfor delay '0:0:15' --
09.10.25 08:25 pHqghUme

is it ok if I upload an image?
09.10.25 08:25 pHqghUme

e
09.10.25 08:25 pHqghUme

e
09.10.25 08:25 pHqghUme

e
09.10.25 08:25 pHqghUme

can I ask you a question please?9IDOn7ik'; waitfor delay '0:0:15' --
09.10.25 08:26 pHqghUme

can I ask you a question please?MQOVJH7P' OR 921=(SELECT 921 FROM PG_SLEEP(15))--
09.10.25 08:26 pHqghUme

e
09.10.25 08:27 pHqghUme

can I ask you a question please?64e1xqge') OR 107=(SELECT 107 FROM PG_SLEEP(15))--
09.10.25 08:27 pHqghUme

can I ask you a question please?ODDe7Ze5')) OR 82=(SELECT 82 FROM PG_SLEEP(15))--
09.10.25 08:28 pHqghUme

can I ask you a question please?'||DBMS_PIPE.RECEIVE_MESSAGE(CHR(98)||CHR(98)||CHR(98),15)||'
09.10.25 08:28 pHqghUme

can I ask you a question please?
09.10.25 08:28 pHqghUme

can I ask you a question please?'"
09.10.25 08:28 pHqghUme

@@olQP6
09.10.25 08:28 pHqghUme

(select 198766*667891)
09.10.25 08:28 pHqghUme

(select 198766*667891 from DUAL)
09.10.25 08:30 pHqghUme

is it ok if I upload an image?
09.10.25 08:33 pHqghUme

can I ask you a question please?
09.10.25 08:34 pHqghUme

can I ask you a question please?
09.10.25 08:34 pHqghUme

if(now()=sysdate(),sleep(15),0)
09.10.25 08:35 pHqghUme

e
09.10.25 08:36 pHqghUme

is it ok if I upload an image?
09.10.25 08:36 pHqghUme

is it ok if I upload an image?
09.10.25 08:37 pHqghUme

is it ok if I upload an image?
09.10.25 08:37 pHqghUme

is it ok if I upload an image?
09.10.25 08:37 pHqghUme

e
09.10.25 08:37 pHqghUme

e
09.10.25 08:40 pHqghUme

can I ask you a question please?
09.10.25 08:40 pHqghUme

is it ok if I upload an image?
09.10.25 08:41 pHqghUme

e
09.10.25 08:41 pHqghUme

can I ask you a question please?
09.10.25 08:42 pHqghUme

can I ask you a question please?
09.10.25 08:42 pHqghUme

is it ok if I upload an image?
09.10.25 08:42 pHqghUme

e
09.10.25 11:05 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
09.10.25 11:05 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
09.10.25 11:05 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
09.10.25 11:05 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
11.10.25 04:41 luciajessy3

Don’t be deceived by different testimonies online that is most likely wrong. I have made use of several recovery options that got me disappointed at the end of the day but I must confess that the tech genius I eventually found is the best out here. It’s better you devise your time to find the valid professional that can help you recover your stolen or lost crypto such as bitcoins rather than falling victim of other amateur hackers that cannot get the job done. ADAMWILSON . TRADING @ CONSULTANT COM / WHATSAPP ; +1 (603) 702 ( 4335 ) is the most reliable and authentic blockchain tech expert you can work with to recover what you lost to scammers. They helped me get back on my feet and I’m very grateful for that. Contact their email today to recover your lost coins ASAP…
11.10.25 10:44 Tonerdomark

A thief took my Dogecoin and wrecked my life. Then Mr. Sylvester stepped in and changed everything. He got back €211,000 for me, every single cent of my gains. His calm confidence and strong tech skills rebuilt my trust. Thanks to him, I recovered my cash with no issues. After months of stress, I felt huge relief. I had full faith in him. If a scam stole your money, reach out to him today at { yt7cracker@gmail . com } His help sparked my full turnaround.
12.10.25 01:12 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
12.10.25 01:12 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
12.10.25 19:53 Tonerdomark

A crook swiped my Dogecoin. It ruined my whole world. Then Mr. Sylvester showed up. He fixed it all. He pulled back €211,000 for me. Not one cent missing from my profits. His steady cool and sharp tech know-how won back my trust. I got my money smooth and sound. After endless worry, relief hit me hard. I trusted him completely. Lost cash to a scam? Hit him up now at { yt7cracker@gmail . com }. His aid turned my life around. WhatsApp at +1 512 577 7957.
12.10.25 21:36 blessing

Writing this review is a joy. Marie has provided excellent service ever since I started working with her in early 2018. I was worried I wouldn't be able to get my coins back after they were stolen by hackers. I had no idea where to begin, therefore it was a nightmare for me. However, things became easier for me after my friend sent me to [email protected] and +1 7127594675 on WhatsApp. I'm happy that she was able to retrieve my bitcoin so that I could resume trading.
13.10.25 01:11 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
13.10.25 01:11 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
14.10.25 01:15 tyleradams

Hi. Please be wise, do not make the same mistake I had made in the past, I was a victim of bitcoin scam, I saw a glamorous review showering praises and marketing an investment firm, I reached out to them on what their contracts are, and I invested $28,000, which I was promised to get my first 15% profit in weeks, when it’s time to get my profits, I got to know the company was bogus, they kept asking me to invest more and I ran out of patience then requested to have my money back, they refused to answer nor refund my funds, not until a friend of mine introduced me to the NVIDIA TECH HACKERS, so I reached out and after tabling my complaints, they were swift to action and within 36 hours I got back my funds with the due profit. I couldn’t contain the joy in me. I urge you guys to reach out to NVIDIA TECH HACKERS on their email: [email protected]
14.10.25 08:46 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
14.10.25 08:46 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
14.10.25 08:46 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: 📧 Email: [email protected] 📱 Telegram: @Capitalcryptorecover Contact: [email protected] 📞 Call/Text: +1 (336) 390-6684 🌐 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
15.10.25 18:07 crypto

Cryptocurrency's digital realm presents many opportunities, but it also conceals complex frauds. It is quite painful to lose your cryptocurrency to scam. You can feel harassed and lost as a result. If you have been the victim of a cryptocurrency scam, this guide explains what to do ASAP. Following these procedures will help you avoid further issues or get your money back. Communication with Marie ([email protected] and WhatsApp: +1 7127594675) can make all the difference.
15.10.25 21:52 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
15.10.25 21:52 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
17.10.25 20:17 tyleradams

As time passes, there are an increasing number of frauds involving Bitcoin and other cryptocurrencies. Although there are many individuals who advertise recovering money online, people should use caution in dealing, especially when money is involved. You can trust NVIDIA TECH HACKERS [[email protected]], I promise. They are the top internet recovery company, and as their names indicate, your money is reclaimed as soon as feasible. My bitcoin was successfully retrieved in large part thanks to NVIDIA TECH HACKERS. Ensure that you get top-notch service; NVIDIA TECH HACKERS provides evidence of its work; and payment is only made when the service has been completed to your satisfaction. Reach them via email: [email protected] on google mail
17.10.25 20:20 lindseyvonn

Have you gotten yourself involved in a cryptocurrency scam or any scam at all? If yes, know that you are not alone, there are a lot of people in this same situation. I'm a Health Worker and was a victim of a cryptocurrency scam that cost me a lot of money. This happened a few weeks ago, there’s only one solution which is to talk to the right people, if you don’t do this you will end up being really depressed. I was really devastated until went on LinkedIn one evening after my work hours and i saw lots of reviews popped up on my feed about [email protected], I sent an email to the team who came highly recommended - [email protected] I started seeing some hope for myself from the moment I sent them an email. The good part is they made the entire process stress free for me, i literally sat and waited for them to finish and I received what I lost in my wallet
17.10.25 20:22 richardcharles

I would recommend NVIDIA TECH HACKERS to anyone that needs this service. I decided to get into crypto investment and I ended up getting my crypto lost to an investor late last year. The guy who was supposed to be managing my account turned out to be a scammer all along. I invested 56,000 USD and at first, my reading and profit margins were looking good. I started getting worried when I couldn’t make withdrawals and realized that I’ve been scammed. I came across some of the testimonials that people said about NVIDIA TECH HACKERS and how helpful he has been in recovering their funds. I immediately contacted him in his mail at [email protected] so I can get his assistance. One week into the recovery process the funds were traced and recovered back from the scammer. I can't appreciate him enough for his professionalism.
17.10.25 20:23 stevekalfman

If you need a hacker for scam crypto recovery or mobile spy access remotely kindly reach out to [email protected] for quick response, I hired this hacker and he did a nice job. before NVIDIA TECH HACKERS, I met with different hacker's online which turns out to be scam, this NVIDIA TECH HACKERS case was different and he is the trusted hacker I can vote and refer.
17.10.25 21:42 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
17.10.25 21:42 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
17.10.25 21:42 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
21.10.25 08:39 debby131

Given how swiftly the cryptocurrency market moves, losing USDT may be a terrifying and upsetting experience. Whether you experienced a technical problem, a transaction error, or were the victim of fraud, it is important to understand the potential recovery routes. Marie can assist you in determining the specific actions you can take to attempt to regain your lost USDT when you need guidance and clarification. You can reach her via email at [email protected] and WhatsApp at +1 7127594675.
21.10.25 11:45 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
21.10.25 11:45 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
22.10.25 04:48 MATT PHILLIP

I never imagined I’d fall for a crypto romance scam but it happened. Over the course of a few months, I sent nearly $150,000 worth of Bitcoin to someone I genuinely believed I was building a future with. When they disappeared without a trace, I was left heartbroken, humiliated, and financially devastated. For a long time, I didn’t tell anyone. I felt ashamed. But eventually, while searching for answers, I came across a Reddit thread that mentioned Agent Jasmine Lopez. I reached out, not expecting much. To my surprise, she treated me with kindness, not judgment. She used advanced tools like blockchain forensics, IP tracing, and smart contract analysis and with persistence and legal support, she was able to recover nearly 85% of what I lost. I know not everyone gets that kind of outcome, but thanks to [email protected] WhatsApp at +44 736-644-5035, I’ve started to reclaim not just my assets, but my confidence and peace of mind. If you’re going through something similar, you’re not alone and there is hope.
22.10.25 07:36 donnacollier

HOW I RECOVERED MY BITCOIN FROM CRYPTO INVESTMENT SCAM HOW I RECOVERED MY BITCOIN FROM CRYPTO INVESTMENT SCAM Hello everyone l’m by name Donna collier I live in urbandale 3 weeks ago was the darkest days of my life, i invested my hard earned money the sum of $123,000 into a crypto currency platform I was introduced to by a friend I met online everything happen so fast I was promised 300% of return of investment when it was time for me to cash out my investment and profit the platform was down I was so diversitated confused and tried to end it all then I came across upswing Ai a renowned expert in crypto currency and digital assets recovery at first it seem impossible after taking up my case within the the next 48 hours they where able to track down those fraud stars and recover my money I can’t recommend them enough to any one facing likewise change I will recommend you contact upswing Ai on Contact Details: WhatsApp +,1,2,0,2,8,1,0,1,4,0,7 E m a i l @Upswing-ai.com Website https: // u psw ing-ai. com/ platform /
22.10.25 11:59 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
22.10.25 11:59 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
22.10.25 16:31 MATT PHILLIP

I never imagined I’d fall for a crypto romance scam but it happened. Over the course of a few months, I sent nearly $150,000 worth of Bitcoin to someone I genuinely believed I was building a future with. When they disappeared without a trace, I was left heartbroken, humiliated, and financially devastated. For a long time, I didn’t tell anyone. I felt ashamed. But eventually, while searching for answers, I came across a Reddit thread that mentioned Agent Jasmine Lopez. I reached out, not expecting much. To my surprise, she treated me with kindness, not judgment. She used advanced tools like blockchain forensics, IP tracing, and smart contract analysis and with persistence and legal support, she was able to recover nearly 85% of what I lost. I know not everyone gets that kind of outcome, but thanks to [email protected] WhatsApp at +44 736-644-5035, I’ve started to reclaim not just my assets, but my confidence and peace of mind. If you’re going through something similar, you’re not alone and there is hope.
23.10.25 01:52 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
23.10.25 01:52 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
23.10.25 15:47 MATT PHILLIP

I never imagined I’d fall for a crypto romance scam but it happened. Over the course of a few months, I sent nearly $150,000 worth of Bitcoin to someone I genuinely believed I was building a future with. When they disappeared without a trace, I was left heartbroken, humiliated, and financially devastated. For a long time, I didn’t tell anyone. I felt ashamed. But eventually, while searching for answers, I came across a Reddit thread that mentioned Agent Jasmine Lopez. I reached out, not expecting much. To my surprise, she treated me with kindness, not judgment. She used advanced tools like blockchain forensics, IP tracing, and smart contract analysis and with persistence and legal support, she was able to recover nearly 85% of what I lost. I know not everyone gets that kind of outcome, but thanks to [email protected] WhatsApp at +44 736-644-5035, I’ve started to reclaim not just my assets, but my confidence and peace of mind. If you’re going through something similar, you’re not alone and there is hope.
23.10.25 21:43 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
23.10.25 21:43 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.10.25 06:08 MATT PHILLIP

I never imagined I’d fall for a crypto romance scam but it happened. Over the course of a few months, I sent nearly $150,000 worth of Bitcoin to someone I genuinely believed I was building a future with. When they disappeared without a trace, I was left heartbroken, humiliated, and financially devastated. For a long time, I didn’t tell anyone. I felt ashamed. But eventually, while searching for answers, I came across a Reddit thread that mentioned Agent Jasmine Lopez. I reached out, not expecting much. To my surprise, she treated me with kindness, not judgment. She used advanced tools like blockchain forensics, IP tracing, and smart contract analysis and with persistence and legal support, she was able to recover nearly 85% of what I lost. I know not everyone gets that kind of outcome, but thanks to [email protected] WhatsApp at +44 736-644-5035, I’ve started to reclaim not just my assets, but my confidence and peace of mind. If you’re going through something similar, you’re not alone and there is hope.
24.10.25 06:40 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.10.25 06:40 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.10.25 06:54 MATT PHILLIP

I never imagined I’d fall for a crypto romance scam but it happened. Over the course of a few months, I sent nearly $150,000 worth of Bitcoin to someone I genuinely believed I was building a future with. When they disappeared without a trace, I was left heartbroken, humiliated, and financially devastated. For a long time, I didn’t tell anyone. I felt ashamed. But eventually, while searching for answers, I came across a Reddit thread that mentioned Agent Jasmine Lopez. I reached out, not expecting much. To my surprise, she treated me with kindness, not judgment. She used advanced tools like blockchain forensics, IP tracing, and smart contract analysis and with persistence and legal support, she was able to recover nearly 85% of what I lost. I know not everyone gets that kind of outcome, but thanks to [email protected] WhatsApp at +44 736-644-5035, I’ve started to reclaim not just my assets, but my confidence and peace of mind. If you’re going through something similar, you’re not alone and there is hope.
24.10.25 18:18 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.10.25 18:18 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
25.10.25 00:49 tyleradams

*HOW TO GET BACK YOUR STOLEN BITCOIN FROM SCAMMERS* ⭐️⭐️⭐️⭐️⭐️ Hello everyone, I can see a lot of things going wrong these last few days of investing online and getting scammed. I was in your shoes when I invested in a bogus binary option and was duped out of $87,000 in BTC, but thanks to the assistance of NVIDIA TECH HACKERS. They helped me get back my BTC. I didn’t trust the Hackers at first, but they were recommended to me by a friend whom I greatly respect. I received my refund in two days. I must recommend NVIDIA TECH HACKERS they are fantastic. 📧 [email protected]
25.10.25 00:50 stevekalfman

If you ever need hacking services, look no further. I found myself in a difficult situation after losing almost $510,000 USD in bitcoin. I was distraught and had no hope of survival because i lost my life savings to this scam, I had no chance of recovering my investment. Until i came across an article on google about ([email protected]). A real-life recovery expert, everything changed completely after contacting him and explaining my ordeal to him and he quickly stepped in and assisted me in recovering 95% of my lost funds. They guarantee their clients and i got the highest level of happiness their services are highly recommended. so I’m putting this here for anyone who require their services too" Email: [email protected]
25.10.25 05:05 victoriabenny463

A few months ago, I fell for what looked like a legitimate online trading platform. The website was sleek, the support team was responsive, and they even showed me fake profit charts that seemed to grow daily. I started small, and after a few test withdrawals worked, I invested more eventually depositing 2 Bitcoin in hopes of long-term returns. But one morning, when I tried to withdraw, everything changed. The platform suddenly locked my account, claiming there were additional fees to release my funds. The support chat stopped responding, and the website started giving error messages. That’s when I realized I’d been scammed. I was devastated. Two Bitcoins wasn’t pocket change, it was my savings. I filed reports online, but everyone told me it was impossible to recover crypto once it’s sent. Then, through a discussion forum, I came across Dexdert Net Recovery, a team specializing in tracing and recovering stolen digital assets. Skeptical but desperate, I reached out. Their response was immediate and professional. They asked for transaction IDs, wallet addresses, screenshots of my correspondence with the fake platform, and any proof of payment I had. Within days, their analysts traced the stolen Bitcoin through multiple wallets using blockchain forensics, They updated me step-by-step, Working with their recovery experts and blockchain partners, Dexdert Net Recovery successfully froze the funds and verified ownership of my wallet. After 2 days of persistence and verification, I finally saw my 2 Bitcoin restored back into my own wallet. I couldn’t believe it. What stood out most wasn’t just that they recovered my funds, but how transparent and supportive the Dexdert team was throughout the process. They educated me on crypto safety, how to spot fake trading sites, and how to secure my digital assets in the future. Dexdert Net Recovery truly lives up to its name. They gave me back not just my Bitcoin, but my peace of mind. Reach Out Information Below Via: Email: ([email protected]) Telegram: (https://t.me/Dexdertprorecovery) WhatsApp: (+1 (859) 609‑4156)
25.10.25 10:13 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
25.10.25 10:13 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
26.10.25 01:03 Christopherbelle

A terrible financial shock hit me hard. I lost $860,000 in a fake crypto scheme. Things looked good until I tried to cash out my earnings. Suddenly, my account vanished into thin air. I told the police many times, but help never showed up. Despair washed over me; I felt totally beaten. Then, I found Sylvester Bryant Intelligence. Right away, they were skilled and honest about the whole thing. They looked deep into my situation. They tracked where the stolen funds went. They fought hard for me every step of the way. To my great surprise, they got my money back. I thought that was impossible. I owe them so much for their hard work and truthfulness. Sylvester Bryant Intelligence restored my peace. If scams took your crypto, contact them now. Contact Info: Yt7cracker@gmail . com WhatsApp: ‪+1 512 577 7957. or +44 7428 662701.
26.10.25 18:09 victoriabenny463

A few months ago, I fell for what looked like a legitimate online trading platform. The website was sleek, the support team was responsive, and they even showed me fake profit charts that seemed to grow daily. I started small, and after a few test withdrawals worked, I invested more eventually depositing 2 Bitcoin in hopes of long-term returns. But one morning, when I tried to withdraw, everything changed. The platform suddenly locked my account, claiming there were additional fees to release my funds. The support chat stopped responding, and the website started giving error messages. That’s when I realized I’d been scammed. I was devastated. Two Bitcoins wasn’t pocket change, it was my savings. I filed reports online, but everyone told me it was impossible to recover crypto once it’s sent. Then, through a discussion forum, I came across Dexdert Net Recovery, a team specializing in tracing and recovering stolen digital assets. Skeptical but desperate, I reached out. Their response was immediate and professional. They asked for transaction IDs, wallet addresses, screenshots of my correspondence with the fake platform, and any proof of payment I had. Within days, their analysts traced the stolen Bitcoin through multiple wallets using blockchain forensics, They updated me step-by-step, Working with their recovery experts and blockchain partners, Dexdert Net Recovery successfully froze the funds and verified ownership of my wallet. After 2 days of persistence and verification, I finally saw my 2 Bitcoin restored back into my own wallet. I couldn’t believe it. What stood out most wasn’t just that they recovered my funds, but how transparent and supportive the Dexdert team was throughout the process. They educated me on crypto safety, how to spot fake trading sites, and how to secure my digital assets in the future. Dexdert Net Recovery truly lives up to its name. They gave me back not just my Bitcoin, but my peace of mind. Reach Out Information Below Via: Email: ([email protected]) Telegram: (https://t.me/Dexdertprorecovery) WhatsApp: (+1 (859) 609‑4156)
27.10.25 11:15 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
27.10.25 11:15 harristhomas7376

"In the crypto world, this is great news I want to share. Last year, I fell victim to a scam disguised as a safe investment option. I have invested in crypto trading platforms for about 10yrs thinking I was ensuring myself a retirement income, only to find that all my assets were either frozen, I believed my assets were secure — until I discovered that my BTC funds had been frozen and withdrawals were impossible. It was a devastating moment when I realized I had been scammed, and I thought my Bitcoin was gone forever, Everything changed when a close friend recommended the Capital Crypto Recover Service. Their professionalism, expertise, and dedication enabled me to recover my lost Bitcoin funds back — more than €560.000 DEM to my BTC wallet. What once felt impossible became a reality thanks to their support. If you have lost Bitcoin through scams, hacking, failed withdrawals, or similar challenges, don’t lose hope. I strongly recommend Capital Crypto Recover Service to anyone seeking a reliable and effective solution for recovering any wallet assets. They have a proven track record of successful reputation in recovering lost password assets for their clients and can help you navigate the process of recovering your funds. Don’t let scammers get away with your hard-earned money – contact Email: [email protected] Phone CALL/Text Number: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
27.10.25 17:08 raymondgonzales

Three Cheers for Wizard Hilton Cyber Tech, Bitcoin Savior Extraordinaire In the ever-evolving world of cryptocurrency, where volatility and uncertainty reign supreme, one individual has emerged as a beacon of hope and innovation - Wizard Hilton Cyber Tech, the self-proclaimed "Bitcoin Savior Extraordinaire." This enigmatic figure, with his uncanny ability to navigate the complexities of the digital asset landscape, has captivated the attention of crypto enthusiasts and skeptics alike. Wizard Hilton Cyber Tech journey began as a humble programmer, tinkering with the underlying blockchain technology that powers Bitcoin. But it was his visionary approach and unwavering commitment to the cryptocurrency's potential that propelled him into the limelight. Through a series of strategic investments, groundbreaking algorithmic trading strategies, and a knack for anticipating market shifts, Wizard Hilton Cyber Tech has managed to consistently outperform even the savviest of Wall Street traders. Wizard Hilton Cyber Tech ability to turn even the most tumultuous of Bitcoin price swings into opportunities for substantial gains has earned him the moniker "Wizard," a testament to his unparalleled mastery of the digital currency realm. But Wizard Hilton Cyber Tech impact extends far beyond personal wealth accumulation - Wizard Hilton Cyber Tech has tirelessly advocated for the widespread adoption of Bitcoin, working tirelessly to educate the public, lobby policymakers, and collaborate with industry leaders to overcome the barriers that have long hindered the cryptocurrency's mainstream acceptance. With infectious enthusiasm and boundless energy, Wizard Hilton Cyber Tech has become a true champion of the Bitcoin cause, inspiring a new generation of crypto enthusiasts to embrace the transformative power of this revolutionary technology. As the digital currency landscape continues to evolve, Wizard Hilton Cyber Tech name will undoubtedly be etched in the annals of cryptocurrency history as a visionary, a trailblazer, and, above all, the Bitcoin Savior Extraordinaire. To get your stolen bitcoin back, reach out to Wizard Hilton Cyber Tech via: Email : wizardhiltoncybertech ( @ ) gmail (. ) com WhatsApp number +18737715701 Good Day.
27.10.25 17:20 fatimanorth

The Federal Trade Commission said that more than $1 billion had been lost to cryptocurrency scams in 2023. Victims frequently lose everything, including hope, trust, and wealth. When your hard-earned money disappears into digital shadows, you feel trapped. Nowadays, cryptocurrency frauds are very common. You can get help from recovery specialist Marie at [email protected] and via WhatsApp at +1 7127594675.
28.10.25 00:55 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
28.10.25 00:55 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
29.10.25 03:43 Christopherbelle

A sudden money crisis struck me. I dropped $82,000 to a phony crypto scam. All seemed fine until I went to pull out my gains. Then, i was unable to get back my hard earned money. I reported it to the police several times. No aid came my way. Deep sadness hit me; I felt crushed. That's when I learned about Sylvester Bryant Intelligence. They acted quick and fair from the start. They checked my case closely. They followed the path of my lost cash. They pushed strong for me at each turn. In shock, they brought my funds back. I had figured it could not happen. I thank them for their effort and honesty. Sylvester Bryant Intelligence gave me calm again. If a scam stole your crypto, reach out to them today. Contact Info: Yt7cracker@gmail . com WhatsApp: +1 512 577 7957 or +44 7428 662701.
29.10.25 10:56 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
29.10.25 10:56 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
30.10.25 09:49 Christopherbelle

I suffered a huge money blow when $80,000 slipped away in a bogus crypto plot. It all looked great at first. Then I went to pull out my gains, and poof—my account was gone. I kept telling the cops about it, but they did zip. I felt shattered and lost. That's when I stumbled on Sylvester Bryant Intelligence. Things turned around fast. Right away, they showed clear steps, sharp know-how, and real drive to fix it. They tracked my missing cash step by step and pushed hard till they got it back. I swear, I figured it was a lost cause. Their straight-up work and trust brought back my calm. If a scam stole your crypto, reach out to them now. yt7cracker@gmail . com | WhatsApp: +1 512 577 7957 or +44 7428 662701
30.10.25 12:03 elizabethrush89

God bless Capital Crypto Recover Services for the marvelous work you did in my life, I have learned the hard way that even the most sensible investors can fall victim to scams. When my USD was stolen, for anyone who has fallen victim to one of the bitcoin binary investment scams that are currently ongoing, I felt betrayal and upset. But then I was reading a post on site when I saw a testimony of Wendy Taylor online who recommended that Capital Crypto Recovery has helped her recover scammed funds within 24 hours. after reaching out to this cyber security firm that was able to help me recover my stolen digital assets and bitcoin. I’m genuinely blown away by their amazing service and professionalism. I never imagined I’d be able to get my money back until I complained to Capital Crypto Recovery Services about my difficulties and gave all of the necessary paperwork. I was astounded that it took them 12 hours to reclaim my stolen money back. Without a doubt, my USDT assets were successfully recovered from the scam platform, Thank you so much Sir, I strongly recommend Capital Crypto Recover for any of your bitcoin recovery, digital funds recovery, hacking, and cybersecurity concerns. You reach them Call/Text Number +1 (336)390-6684 His Email: [email protected] Contact Telegram: @Capitalcryptorecover Via Contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1