Этот сайт использует файлы cookies. Продолжая просмотр страниц сайта, вы соглашаетесь с использованием файлов cookies. Если вам нужна дополнительная информация, пожалуйста, посетите страницу Политика файлов Cookie

Прямой эфир

Русский

English

Войти / Регистрация

Cryptocurrencies: 8166 / Markets: 114434

Market Cap: $ 2 766 072 159 155 / 24h Vol: $ 117 489 825 481 / BTC Dominance: 58.763434733426%

Н Новости

Градиентный бустинг. Реализация с нуля на Python и разбор особенностей его модификаций (XGBoost, CatBoost, LightGBM)

На сегодняшний день градиентный бустинг (gradient boosting machine) является одним из основных production-решений при работе с табличными, неоднородными данными, поскольку обладает высокой производительностью и точностью, а если быть точнее, то его модификации, речь о которых пойдёт чуть позже.

В данной статье представлена не только реализация градиентного бустинга GBM с нуля на Python, но а также довольно подробно описаны ключевые особенности его наиболее популярных модификаций.

Ноутбук с данными алгоритмами можно загрузить на Kaggle (eng) и GitHub (rus).

Содержание

Gradient Boosting Machine

Gradient Boosting Machine

Как и в случае с Adaboost, градиентный бустинг добавляет базовые модели в ансамбль последовательно, однако вместо обучения моделей с учётом весов на основе ошибок предшественников, в данном случае модели обучаются на остаточных ошибках (residual errors), допущенных предыдущими моделями.

Принцип работы градиентного бустинга для регрессии

Алгоритм строится следующим образом:

1) первоначальному прогнозу присваивается среднее значение y_train для всех образцов;
2) рассчитываются остатки модели на основе антиградиента функции потерь;
3) регрессионное дерево обучается на X_train и остатках, далее делается прогноз на X_train;
4) полученный прогноз добавляется к первоначальному и шаги 2-4 повторяются для каждого дерева;
5) после обучения всех моделей снова создаётся первоначальный прогноз из шага 1;
6) далее делаются прогнозы для X_test на обученных деревьях и добавляются к первоначальному;
7) полученная сумма и будет конечным прогнозом.

Формулы для расчётов

Функция потерь: $L(y_i, F(x_i)) = \frac{1}{2} (y_i - F(x_i))^2$

Остатки: $r_{ik} = -\left[\frac{\partial L(y_i, F(x_i))}{\partial F(x_i)}\right]_ {F(x) = F_{m-1}(x)} = y_{i} - F_{m-1}(x_i)$

Принцип работы градиентного бустинга для классификации

В случае классификации дела обстоят немного сложнее: к каждому уникальному классу необходимо применить one-hot encoding и перевести в вероятности с помощью softmax-функции, а к прогнозам деревьев добавить коэффициент $\gamma$ , который регулирует степень вклада каждого нового дерева в общую модель для снижения переобучения. Также качество прогнозов градиентного бустинга можно существенно улучшить, применив концепцию K-class LogitBoost: для каждого дерева рассчитываются веса, а только потом на их основе остатки.

Алгоритм строится следующим образом:

1) для y_train применяется one-hot encoding и первоначальному прогнозу приcваиваются значения 0 для каждого класса;
2) прогнозы преобразуются в вероятности с помощью softmax-функции;
3) рассчитываются остатки модели на основе антиградиента функции потерь и вероятностей (а также веса в случае LogitBoost);
4) регрессионное дерево обучается на X_train и остатках (а в случае LogitBoost ещё присваиваются веса), далее делается прогноз на X_train;
5) для каждого листа в дереве рассчитываются коэффициенты $\gamma$ на основе остатков, взятых по позициям наблюдений, попавших в определённый в листовой узел;
6) полученные прогнозы для каждого класса и сумма коэффициентов $\gamma$ добавляются к первоначальным;
7) шаги 2-6 повторяются для каждого дерева в каждом классе;
8) после обучения всех моделей создаётся первоначальный прогноз из шага 1;
9) далее делаются прогнозы для X_test на обученных деревьях по каждому классу и добавляются к первоначальным;
10) классы с максимальной суммой и будут конечным прогнозом.

Формулы для расчётов

Функция потерь:

$L(\{y_k, F_k(x)\}_{1}^{K}) = -\sum\limits_{k=1}^{K} y_k \ log \ p_k(x)$

Вероятность каждого класса:

$p_k(x) = softmax(F_k(x)) = \frac{e^{F_k(x)}}{\sum\limits_{l=1}^{K} e^{F_l(x)}}$

Остатки:

$r_{ik} = -\left[\frac{\partial L(\{y_{il}, F_l(x_i)\}_{l=1}^{K})}{\partial F_k(x_i)}\right]_ {\{F_l(x) = F_{l,m-1}(x)\}_{1}^{K}} = y_{ik} - p_{k,m-1}(x_i)$

Коэффициент $\gamma$ :

$\gamma_{jkm} = \frac{K-1}{K} \frac{\sum_{x_i \in R_{jkm}} r_{ik}} {\sum_{x_i \in R_{jkm}} |r_{ik}| (1 - |r_{ik}|)}$

Веса для K-class LogitBoost:

Остатки для K-class LogitBoost:

$r_{ik} = \frac{K-1}{K} \frac{y_{ik} - p_k(x_i)}{w_k(x_i)}$

Импорт необходимых библиотек

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_diabetes
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, mean_absolute_percentage_error
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import GradientBoostingClassifier, GradientBoostingRegressor
from sklearn.ensemble import HistGradientBoostingClassifier, HistGradientBoostingRegressor
from mlxtend.plotting import plot_decision_regions

Реализация на Python с нуля

class GBMClassifier:
    def __init__(self, logitboost=False, learning_rate=0.1, n_estimators=100,
                 max_depth=3, random_state=0):
        self.logitboost = logitboost
        self.learning_rate = learning_rate
        self.n_estimators = n_estimators
        self.max_depth = max_depth
        self.random_state = random_state

    def _softmax(self, predictions):
        exp = np.exp(predictions)

        return exp / np.sum(exp, axis=1, keepdims=True)

    def _compute_gammas(self, residuals, leaf_indexes, eps=1e-4):
        gammas = []

        for j in np.unique(leaf_indexes):
            x_i = np.where(leaf_indexes == j)
            numerator = np.sum(residuals[x_i])
            norm_residuals_xi = np.linalg.norm(residuals[x_i]) + eps
            denominator = np.sum(norm_residuals_xi * (1 - norm_residuals_xi))
            gamma = (self.K - 1) / self.K * numerator / denominator
            gammas.append(gamma)

        return gammas

    def fit(self, X, y):
        self.K = len(np.unique(y))
        self.trees = {k: [] for k in range(self.K)}
        one_hot_y = pd.get_dummies(y).to_numpy()   # one-hot encoding
        predictions = np.zeros(one_hot_y.shape)

        for _ in range(self.n_estimators):
            probabilities = self._softmax(predictions)

            for k in range(self.K):
                if self.logitboost:   # based on K-class LogitBoost
                    numerator = (one_hot_y.T[k] - probabilities.T[k])
                    denominator = probabilities.T[k] * (1 - probabilities.T[k])
                    residuals = (self.K - 1) / self.K * numerator / denominator
                    weights = denominator
                else:
                    residuals = one_hot_y.T[k] - probabilities.T[k]
                    weights = None

                tree = DecisionTreeRegressor(criterion='friedman_mse', max_depth=self.max_depth,
                                             random_state=self.random_state)
                tree.fit(X, residuals, sample_weight=weights)
                self.trees[k].append(tree)

                leaf_indexes = tree.apply(X)
                gammas = [] if self.logitboost else self._compute_gammas(residuals, leaf_indexes)
                predictions.T[k] += self.learning_rate * tree.predict(X) + np.sum(gammas)

    def predict(self, samples):
        predictions = np.zeros((len(samples), self.K))

        for i in range(self.n_estimators):
            for k in range(self.K):
                predictions.T[k] += self.learning_rate * self.trees[k][i].predict(samples)

        return np.argmax(predictions, axis=1)

class GBMRegressor:
    def __init__(self, learning_rate=0.1, n_estimators=100, max_depth=3, random_state=0):
        self.learning_rate = learning_rate
        self.n_estimators = n_estimators
        self.max_depth = max_depth
        self.random_state = random_state
        self.trees = []

    def fit(self, X, y):
        self.initial_leaf = y.mean()
        predictions = np.zeros(len(y)) + self.initial_leaf

        for _ in range(self.n_estimators):
            residuals = y - predictions
            tree = DecisionTreeRegressor(criterion='friedman_mse', max_depth=self.max_depth,
                                         random_state=self.random_state)
            tree.fit(X, residuals)
            predictions += self.learning_rate * tree.predict(X)
            self.trees.append(tree)

    def predict(self, samples):
        predictions = np.zeros(len(samples)) + self.initial_leaf

        for i in range(self.n_estimators):
            predictions += self.learning_rate * self.trees[i].predict(samples)

        return predictions

Код для отрисовки решающих границ

def decision_boundary_plot(X, y, X_train, y_train, clf, feature_indexes, title=None):
    feature1_name, feature2_name = X.columns[feature_indexes]
    X_feature_columns = X.values[:, feature_indexes]
    X_train_feature_columns = X_train.values[:, feature_indexes]
    clf.fit(X_train_feature_columns, y_train.values)

    plot_decision_regions(X=X_feature_columns, y=y.values, clf=clf)
    plt.xlabel(feature1_name)
    plt.ylabel(feature2_name)
    plt.title(title)

Загрузка датасетов

Для обучения моделей будет использован Glass Classification датасет, где необходимо верно определить тип стекла по его признакам. В случае регрессии используется Diabetes датасет из scikit-learn.

Данные ниже будут использоваться в примерах для всех модификаций градиентного бустинга.

df_path = "/content/drive/MyDrive/glass.csv"
glass_df = pd.read_csv(df_path)
X1, y1 = glass_df.iloc[:, :-1], glass_df.iloc[:, -1]
y1 = pd.Series(LabelEncoder().fit_transform(y1))
X1_train, X1_test, y1_train, y1_test = train_test_split(X1, y1, random_state=0)
print(glass_df)


          RI     Na    Mg    Al     Si     K    Ca    Ba   Fe  Type
0    1.52101  13.64  4.49  1.10  71.78  0.06  8.75  0.00  0.0     1
1    1.51761  13.89  3.60  1.36  72.73  0.48  7.83  0.00  0.0     1
2    1.51618  13.53  3.55  1.54  72.99  0.39  7.78  0.00  0.0     1
3    1.51766  13.21  3.69  1.29  72.61  0.57  8.22  0.00  0.0     1
4    1.51742  13.27  3.62  1.24  73.08  0.55  8.07  0.00  0.0     1
..       ...    ...   ...   ...    ...   ...   ...   ...  ...   ...
209  1.51623  14.14  0.00  2.88  72.61  0.08  9.18  1.06  0.0     7
210  1.51685  14.92  0.00  1.99  73.06  0.00  8.40  1.59  0.0     7
211  1.52065  14.36  0.00  2.02  73.42  0.00  8.44  1.64  0.0     7
212  1.51651  14.38  0.00  1.94  73.61  0.00  8.48  1.57  0.0     7
213  1.51711  14.23  0.00  2.08  73.36  0.00  8.62  1.67  0.0     7

[214 rows x 10 columns]

X2, y2 = load_diabetes(return_X_y=True, as_frame=True)
X2_train, X2_test, y2_train, y2_test = train_test_split(X2, y2, random_state=0)
print(X2, y2, sep='\n')


          age       sex       bmi        bp        s1        s2        s3  \
0    0.038076  0.050680  0.061696  0.021872 -0.044223 -0.034821 -0.043401   
1   -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163  0.074412   
2    0.085299  0.050680  0.044451 -0.005670 -0.045599 -0.034194 -0.032356   
3   -0.089063 -0.044642 -0.011595 -0.036656  0.012191  0.024991 -0.036038   
4    0.005383 -0.044642 -0.036385  0.021872  0.003935  0.015596  0.008142   
..        ...       ...       ...       ...       ...       ...       ...   
437  0.041708  0.050680  0.019662  0.059744 -0.005697 -0.002566 -0.028674   
438 -0.005515  0.050680 -0.015906 -0.067642  0.049341  0.079165 -0.028674   
439  0.041708  0.050680 -0.015906  0.017293 -0.037344 -0.013840 -0.024993   
440 -0.045472 -0.044642  0.039062  0.001215  0.016318  0.015283 -0.028674   
441 -0.045472 -0.044642 -0.073030 -0.081413  0.083740  0.027809  0.173816   

           s4        s5        s6  
0   -0.002592  0.019907 -0.017646  
1   -0.039493 -0.068332 -0.092204  
2   -0.002592  0.002861 -0.025930  
3    0.034309  0.022688 -0.009362  
4   -0.002592 -0.031988 -0.046641  
..        ...       ...       ...  
437 -0.002592  0.031193  0.007207  
438  0.034309 -0.018114  0.044485  
439 -0.011080 -0.046883  0.015491  
440  0.026560  0.044529 -0.025930  
441 -0.039493 -0.004222  0.003064  

[442 rows x 10 columns]
0      151.0
1       75.0
2      141.0
3      206.0
4      135.0
       ...  
437    178.0
438    104.0
439    132.0
440    220.0
441     57.0
Name: target, Length: 442, dtype: float64

Обучение моделей и оценка полученных результатов

В случае классификации ручная реализация немного отличается в плане точности от scikit-learn — это связано с тем, что в последней применяется более усложнённый подход для прогнозирования и расчёта остатков, но суть остаётся прежней, и в большинстве случаев работает примерно также как и ручная реализация, особенно в сравнении с LogitBoost модификацией.

В случае регрессии, ошибка в ручной реализации даже немного ниже, чем в scikit-learn — различия незначительны.

Также стоит отметить, что в scikit-learn есть особая версия градиентного бустинга — HistGradientBoosting на основе LightGBM, предназначенная для работы с большими наборами данных, которая отрабатывает гораздо быстрее и чаще лучше, чем классический градиентный бустинг.

Полученные результаты приведены ниже.

GBMClassifier

gbc = GBMClassifier(random_state=0)
gbc.fit(X1_train, y1_train)
gbc_pred_res = gbc.predict(X1_test)
gbc_accuracy = accuracy_score(gbc_pred_res, y1_test)
print(f'gbc accuracy: {gbc_accuracy}')
print(gbc_pred_res)


gbc accuracy: 0.6111111111111112
[5 0 1 4 1 1 0 1 1 1 1 0 1 1 1 5 0 1 1 1 5 0 5 5 1 0 5 0 1 0 0 4 1 0 0 0 0
 0 0 5 1 4 1 0 0 1 0 1 0 1 0 3 5 0]

GBMClassifier с концепцией LogitBoost

logit_bc = GBMClassifier(logitboost=True, random_state=0)
logit_bc.fit(X1_train, y1_train)
logit_bc_pred_res = logit_bc.predict(X1_test)
logit_bc_accuracy = accuracy_score(logit_bc_pred_res, y1_test)
print(f'logit_gbc accuracy: {logit_bc_accuracy}')
print(logit_bc_pred_res)


logit_gbc accuracy: 0.6666666666666666
[5 0 1 4 1 1 0 1 1 1 1 0 0 1 1 5 2 1 1 0 3 0 5 1 2 0 5 0 0 1 0 0 1 0 0 0 0
 2 0 5 1 4 1 0 1 1 0 1 0 1 0 1 5 0]

GradientBoostingClassifier (scikit-learn)

sk_gbc = GradientBoostingClassifier(random_state=0)
sk_gbc.fit(X1_train, y1_train)
sk_gbc_pred_res = sk_gbc.predict(X1_test)
sk_gbc_accuracy = accuracy_score(sk_gbc_pred_res, y1_test)
print(f'sk_gbc accuracy: {sk_gbc_accuracy}')
print(sk_gbc_pred_res)

feature_indexes = [1, 3]
title1 = 'GradientBoostingClassifier surface'
decision_boundary_plot(X1, y1, X1_train, y1_train, sk_gbc, feature_indexes, title1)


sk_gbc accuracy: 0.6666666666666666
[5 0 1 4 1 1 0 1 1 1 1 0 1 1 1 5 2 1 2 0 1 0 5 5 0 0 5 0 0 1 0 0 1 0 1 0 0
 0 0 5 1 1 1 0 1 1 0 1 0 1 0 1 5 0]

HistGradientBoostingClassifier (scikit-learn)

sk_hist_gbc = HistGradientBoostingClassifier(random_state=0)
sk_hist_gbc.fit(X1_train, y1_train)
sk_hist_gbc_pred_res = sk_hist_gbc.predict(X1_test)
sk_hist_gbc_accuracy = accuracy_score(sk_hist_gbc_pred_res, y1_test)
print(f'sk_hist_gbc accuracy: {sk_hist_gbc_accuracy}')
print(sk_hist_gbc_pred_res)

feature_indexes = [1, 3]
title2 = 'HistGradientBoostingClassifier surface'
decision_boundary_plot(X1, y1, X1_train, y1_train, sk_hist_gbc, feature_indexes, title2)


sk_hist_gbc accuracy: 0.7407407407407407
[5 0 1 4 1 1 0 1 1 1 0 1 0 1 1 5 2 1 0 2 3 0 5 5 0 0 5 0 1 1 0 0 1 0 0 0 0
 2 0 5 1 4 1 0 0 1 0 1 0 1 0 1 5 0]

GBMRegressor

gbr = GBMRegressor(random_state=0)
gbr.fit(X2_train, y2_train)
gbr_pred_res = gbr.predict(X2_test)
mape = mean_absolute_percentage_error(gbr_pred_res, y2_test)
print(f'gbr mape: {mape}')
print(gbr_pred_res)


gbr mape: 0.3422881238725283
[255.26058697 236.87383554 189.22369747 111.4291292  195.31958776
 249.2293251   97.46956406 205.91110801 133.47952304 240.88679883
 187.19018483 171.09963906 132.63933991  86.28964353 286.56077589
 105.78347251 161.94686043  72.58069452 105.92470096 236.52718059
 178.31146418 127.93036968 164.16563703 147.68681371 229.72747212
 194.16656023 142.44300193  68.79177464 237.21638657 163.36730299
 228.0338107   85.82843049 122.42781509 153.99332065 133.44848601
 159.00754731 151.98981608 147.1238119   88.56372077 200.84785209
 111.32129613 157.60108633 137.94395538 173.39402287 176.29284757
  62.85019316 109.646804   124.14982054  83.45697583 266.40618239
 141.2307366   42.05033124 163.57898214 149.92213387 230.37326368
 226.98842404 176.95426024 128.73302071  94.7641018  180.13853236
 242.31821245 147.52903035 130.45931832  97.32617479 246.70709949
 155.26338705  88.36785447 243.39330218 215.93042351  57.89243255
  87.76236479 138.21222825 101.23449416 140.80521012 128.63575744
 180.15062966 126.14000544 220.28212417 272.05642136 203.4384734
 135.96206295 180.87259854  55.62066017 241.84218009 137.19320157
  85.57441147 154.16763601 198.15465153 107.34659467 158.72672412
 102.17554103 114.4602227   70.44016611 165.66282053 108.02991765
  87.88855231 218.47152891 250.1208086  131.93276827 164.8880371
 174.16393905 119.21512394 150.9405975   93.44225978 234.7444088
 139.46581273 248.28728947 276.8291188  121.36081625  77.12248695
 252.1021757 ]

GradientBoostingRegressor (scikit-learn)

sk_gbr = GradientBoostingRegressor(random_state=0)
sk_gbr.fit(X2_train, y2_train)
sk_gbr_pred_res = sk_gbr.predict(X2_test)
sk_mape = mean_absolute_percentage_error(sk_gbr_pred_res, y2_test)
print(f'sk_gbr mape: {sk_mape}')
print(sk_gbr_pred_res)


sk_gbr mape: 0.34315075601365186
[255.26058697 236.87383554 189.22369747 111.4291292  195.31958776
 249.2293251   97.46956406 205.91110801 133.47952304 240.88679883
 187.19018483 171.09963906 132.63933991  86.28964353 286.56077589
 105.78347251 161.94686043  72.58069452 105.92470096 236.52718059
 182.08417504 127.93036968 164.16563703 147.68681371 229.72747212
 194.16656023 142.44300193  68.79177464 237.21638657 163.36730299
 223.14955275  85.82843049 122.42781509 153.99332065 133.44848601
 159.00754731 151.98981608 147.1238119   88.56372077 200.84785209
 117.71059889 157.60108633 137.94395538 173.39402287 176.29284757
  62.85019316 109.646804   124.14982054  83.45697583 266.40618239
 141.2307366   42.05033124 163.57898214 149.92213387 230.37326368
 226.98842404 176.95426024 128.73302071  94.7641018  180.13853236
 242.31821245 147.52903035 131.71442058  97.32617479 246.70709949
 155.26338705  88.36785447 243.39330218 215.93042351  54.04270734
  87.76236479 138.21222825 101.23449416 140.80521012 128.63575744
 180.15062966 126.14000544 220.28212417 272.05642136 203.4384734
 135.96206295 180.87259854  55.62066017 241.84218009 137.19320157
  85.57441147 154.16763601 198.15465153 107.34659467 158.72672412
 102.17554103 114.4602227   70.44016611 165.66282053 108.02991765
  87.88855231 218.47152891 247.93843617 131.93276827 164.8880371
 174.16393905 119.21512394 150.9405975   93.44225978 234.7444088
 139.46581273 248.28728947 286.90898175 121.36081625  77.12248695
 252.1021757 ]

HistGradientBoostingRegressor (scikit-learn)

sk_hist_gbr = HistGradientBoostingRegressor(random_state=0)
sk_hist_gbr.fit(X2_train, y2_train)
sk_hist_gbr_pred_res = sk_hist_gbr.predict(X2_test)
sk_hist_mape = mean_absolute_percentage_error(sk_hist_gbr_pred_res, y2_test)
print(f'sk_hist_gbr mape: {sk_hist_mape}')
print(sk_hist_gbr_pred_res)


sk_hist_gbr mape: 0.3625770694795175
[264.59007545 243.59190524 180.62442773 108.53165744 195.17695513
 258.1710931  112.67894454 239.18006052 144.50027011 244.02461776
 180.19087697 171.2628973  142.6966707   94.26236013 301.44493762
  90.25622203 154.82566511  75.43291346  96.36987316 230.14565012
 196.50928346 134.41863815 167.28719774 118.51218378 209.75431111
 176.46248808 113.40188129  68.47515467 248.57638999 116.15751993
 195.54397416  59.94733366 153.03012884 165.84819847 135.95402508
 133.54279177 190.26058998 153.08963938  86.65214731 185.07312005
 110.00197459 142.99434304 111.32566034 205.63519803 163.76035901
  68.26003406 118.15074567 101.01137587  57.54356002 247.29839792
 140.32714196  41.87878548 149.89720974 189.08931404 251.48016218
 175.76119949 160.38118455 110.76716142 107.00114228 154.75409577
 256.3194536  148.8935956  119.32752592  78.65118163 252.18315433
 122.833807    92.10033912 219.56310647 199.35553751  87.27114699
  78.83529151 114.14943246  86.07433503 110.44443265 126.1152782
 193.58960355 125.06171193 207.62012728 297.41734601 206.57210109
 121.92170504 194.37322797  69.66978916 260.97214455  99.77944002
 104.34927662 131.51094535 200.0857846  114.70076625 144.92225838
 107.08233694  84.70304021  65.20782844 156.77203859  93.87016441
 117.10361355 213.79169982 258.68468943 129.45477593 140.06443861
 176.72311312 107.95798342 172.45179867 115.75603211 240.83612061
 102.2339954  249.6969811  293.80624063 110.47825419  73.86099524
 238.42985813]

Преимущества и недостатки градиентного бустинга (GBM)

Преимущества:

высокая точность;
хорошо работает с неоднородными данными;
лежит в основе других модификаций градиентного бустинга.

Недостатки:

низкая скорость обучения при работе с большими датасетами;
отсутствие возможности работы с категориальными признаками;
возможность переобучения из-за плохой предобработки данных.

Стоит отметить, что вышеперечисленные недостатки были устранены частично либо полностью в модификациях градиентного бустинга, про которые сейчас и пойдёт речь.

Дополнительные источники

Статья «Greedy Function Approximation: A Gradient Boosting Machine», Jerome H. Friedman.

Документация:

Лекции:

Пошаговое построение Gradient Boosting:

Особенности XGBoost

XGBoost (Extreme Gradient Boosting) является, пожалуй, самой популярной модификацией градиентного бустинга на сегодняшний день, с помощью которой было выиграно множество соревнований на Kaggle. Рассмотрим причины по которым так произошло.

В отличие от GBM в XGboost изначальный прогноз регрессионного дерева устанавливается f_0 (x) = 0.5 и вместо пней с глубиной 3 берутся более глубокие деревья с листьями в диапазоне от 8 до 32 как в случае регрессии, так и классификации, а к функции потерь добавляется регуляризация (L1 или L2 на выбор), которая штрафует дерево с высокой суммой нормы значений в листьях. Данная сумма представлена в виде целевой функции, расчёт значений которой аппроксимируется разложением в ряд Тейлора 2-го порядка.

Формулы для расчётов

Целевая функция:

$Obj^{\ t} = \sum\limits_{i=1}^{N} L(y_i, \hat{y_i}^{(t-1)} + f_t(x_i)) + \Omega(f_t)$

L2-регуляризация:

$\Omega(f_t) = \gamma \ T + \frac{1}{2} \lambda||w||^2$

Аппроксимация целевой функции разложением в ряд Тейлора 2-го порядка:

$Obj^{\ t} \approx \sum\limits_{i=1}^{N} [L(y_i, \hat{y_i}^{(t-1)}) + g_i f_t(x_i) + \frac{1}{2} h_i f_t^2(x_i) ] + \gamma \ T + \frac{1}{2} \lambda \sum\limits_{j=1}^{T} w_j^2 = \\ = \sum\limits_{j=1}^{T} [(\sum\limits_{i \in I_j}g_i) w_j + \frac{1}{2} (\sum\limits_{i \in I_j}h_i + \lambda) w_j^2 ] + \gamma \ T$

Где:

градиент первого порядка: ;
градиент второго порядка (гессиан): ;
параметры регуляризации: $\gamma, \lambda$ .

Приближение второго порядка удобно с вычислительной точки зрения, поскольку большинство членов одинаковы на каждой отдельной итерации и большая часть выражения может быть вычислена один раз и повторно использована как константа для всех остальных разбиений, и в таком случае единственное, что остаётся вычислить — это f_t(x_i) и $\Omega(f_t)$ .

Оптимальное значение веса j-го листового узла в дереве рассчитывается следующим образом:

$w^*_j = -\frac{\sum_{i \in I_j} g_i}{\sum_{i \in I_j} h_i + \lambda}$

Мера загрязнённости представляет из себя качество структуры дерева q и рассчитывается следующим образом:

$Obj^{\ t} = -\frac{1}{2} \sum\limits_{j=1}^{T} \frac{(\sum_{i \in I_j} g_i)^2}{\sum_{i \in I_j} h_i + \lambda} + \gamma \ T$

Чем меньше это значение, тем лучше структура дерева.

Поскольку перечисление всевозможных древовидных структур и выбор наилучшей весьма осложнён, мы можем оптимизировать данную задачу, добавляя по одному разбиению узла дерева за раз. В частности, мы итеративно добавляем ветви к дереву, используя жадный алгоритм, а информационный прирост будет выглядеть следующим образом:

$gain = \frac{1}{2} \left[\frac{(\sum_{i \in I_L} g_i)^2}{\sum_{i \in I_L} h_i + \lambda} + \frac{(\sum_{i \in I_R} g_i)^2}{\sum_{i \in I_R} h_i + \lambda} - \frac{(\sum_{i \in I} g_i)^2}{\sum_{i \in I} h_i + \lambda} \right] - \gamma$

Информационный прирост будет показывать изменение чистоты узла после его разбиения. Его максимальное значение будет соответствовать наилучшему пороговому значению (критерию разбиения) узла в дереве.

Стоит также упомянуть, что в XGBoost поддерживается выбор линейных моделей в качестве базовых, но, в целом, они работают хуже деревьев.

Дополнительные оптимизации в XGBoost

Weighted quantile sketch

В случае работы с большими наборами данных в XGBoost используется приближённый жадный алгоритм. Это означает, что вместо тестирования всех возможных пороговых значений мы тестируем только квантили. Используя распараллеливание для разделения набора данных и создания ветвей в каждом дереве независимо друг от друга чтобы несколько ядер могли работать с ними одновременно, алгоритм взвешенных квантилей (weighted quantile sketch) объединяет данные в приближённую гистограмму, разделённую на взвешенные квантили, которые помещают наблюдения с прогнозами низкой достоверности в квантили с меньшим количеством наблюдений.

Cache-aware access & out-of-core computing

Для более быстрого вычисления прогнозов градиенты и гессианы хранятся в кэш-памяти (cache-aware access), а в случае с датасетами большого размера, когда недостаточно кэш и оперативной памяти, задействуется память из жёсткого диска. Учитывая, что чтение данных на жёстком диске выполняется гораздо медленнее чем в кэш-памяти, для ускоренной обработки производится их компрессия. Данные хранятся в единицах памяти, называемых блоками, информация в которых хранится в формате сжатых столбцов, отсортированных по соответствующим значениям параллельно с использованием всех доступных ядер ЦП, а распаковка каждого блока осуществляется в независимом потоке при загрузке в память. Данная техника называется "вычисление вне ядра" (out-of-core computing). При наличии нескольких жёстких дисков, применяется параллельное сегментирование блоков для каждого жёсткого диска (sharding).

При использовании сжатых данных необходимо учитывать, что процесс сжатия и распаковки занимает дополнительное время, поэтому выбор алгоритма сжатия (с потерями и без) и его параметров может повлиять на производительность в целом, однако это всё равно значительно быстрее чтения без сжатия.

Sparsity-aware split finding

В XGboost также поддерживается работа с пропусками. Суть заключается в следующем: из исходного датасета убираются наблюдения с пропусками и представляются как отдельный датасет. После построения дерева для наблюдений без пропусков рассчитываются их остатки, далее — остатки для наблюдений с пропусками, которые присоединяются к остаткам дерева как для левого, так и для правого узла. В конце рассчитывается информационный прирост (gain) для каждого узла в дереве: выбирается порог для разбиения с максимальным приростом. Данная техника называется "поиск разбиения с учётом разряженности" (sparsity-aware split finding).

Обучение на GPU

Для ускорения процесса обучения в XGBoost реализована поддержка GPU с помощью параметров device="cuda" и tree_method="gpu_hist". Также стоит отметить, что в данном алгоритме поддерживается полностью распределённое обучение на GPU с использованием Dask, Spark и PySpark.

Пакетная реализация XGBoost и оценка полученных результатов

Благодаря вышеуказанным оптимизациям и особенностям XGBoost работает лучше классической реализации, что видно на примере классификации, однако на примере регрессии в данном случае результат оказался хуже. Если хорошо подготовить данные и поэкспериментировать с настройкой гиперпараметров, XGBoost будет работать гораздо лучше как в случае регрессии, так и классификации.

XGBClassifier

from xgboost import XGBClassifier, XGBRegressor

xgbc = XGBClassifier(seed=0)
xgbc.fit(X1_train, y1_train)
xgbc_pred_res = xgbc.predict(X1_test)
xgbc_accuracy = accuracy_score(xgbc_pred_res, y1_test)
print(f'xgbc accuracy: {xgbc_accuracy}')
print(xgbc_pred_res)

feature_indexes = [1, 3]
title2 = 'XGBClassifier surface'
decision_boundary_plot(X1, y1, X1_train, y1_train, xgbc, feature_indexes, title2)


xgbc accuracy: 0.6851851851851852
[5 0 1 4 1 1 0 1 1 1 3 0 1 1 1 5 2 1 1 2 3 0 5 5 0 0 5 0 0 1 0 0 1 0 0 0 0
 0 0 5 1 4 1 0 1 1 0 1 0 1 0 1 5 0]

XGBRegressor

xgbr = XGBRegressor(seed=0)
xgbr.fit(X2_train, y2_train)
xgbr_pred_res = xgbr.predict(X2_test)
xgbr_mape = mean_absolute_percentage_error(xgbr_pred_res, y2_test)
print(f'xgbr mape: {xgbr_mape}')
print(xgbr_pred_res)


xgbr mape: 0.3781388070785457
[281.3304   241.21165  212.13124  136.89372  144.54208  261.7871
 125.257454 236.16089  199.79964  250.18103  188.50882  126.724464
 161.2112    73.98386  302.50272   97.35103  160.85774   67.10394
  96.0406   284.72873  181.48717  117.764626 182.91393  127.76686
 229.30229  173.58592  170.03932   76.73085  240.7698   117.70699
 224.36049   63.628925 124.27327  142.07358  173.09418  139.11557
 163.61208   92.8561    95.91142  217.65016  112.03522  122.800545
 110.06953  181.34201  159.39062   61.97588  114.534805 145.20259
  84.905815 284.77567  195.29976   45.550606  92.54941  180.4689
 279.12097  241.1343   177.77785  130.43887  159.53189  158.2313
 253.25339  181.86093  120.71679   90.604706 274.5059   140.21674
  92.80314  211.71936  233.20622  110.791824  67.89235  133.62137
 123.05923  102.70884  167.2115   159.32262  131.7305   243.41493
 254.97531  210.0554   130.00981  222.21094   52.864147 251.23187
 118.602684  87.89112  121.147095 218.15724   90.9274    91.69777
 101.95428   85.79966   79.59963  155.879    118.25745   99.5975
 231.70824  189.6652   121.97507  147.51166  152.10051  102.239395
 177.74615  107.55845  271.80927  121.41044  232.97308  292.89096
 129.39447   60.501278 259.46295 ]

Преимущества и недостатки XGBoost

Преимущества:

относительно высокая скорость обучения и точность прогнозов;
возможность работы с пропусками;
гибкость в конфигурации из-за большого количества гиперпараметров.

Недостатки:

сложность в конфигурации из-за большого количества гиперпараметров;
на данный момент отсутствует полноценная (присутствует экспериментальная) возможность работы с категориальными признаками, что может сильно снизить производительность при кодировании больших объёмов данных.

Дополнительные источники

Статья «XGBoost: A Scalable Tree Boosting System», Tianqi Chen, Carlos Guestrin.

Документация.

Лекции:

один;
два.

Пошаговое построение XGBoost:

Особенности CatBoost

CatBoost (Categorical Boosting) — более эффективная реализация градиентного бустинга от компании Яндекс, особенностью которой является возможность работы с категориальными признаками без их предварительной обработки. Разберёмся подробнее как это происходит.

Ordered target (CatBoost) encoding

При кодировании категориальных признаков к бустингам применяется один из нескольких классических подходов:

target encoding;
one-hot encoding (хорошо работает лишь в том случае, когда в признаках содержится небольшое количество значений);
feature hashing в несколько корзин (исходные данные преобразуются в уникальную последовательность символов (хэш-значения), которые разбиваются на группы (корзины) и обрабатываются параллельно с последующим применением target или one-hot encoding);
mean target encoding (рассчитывается новое значение для текущего категориального признака как среднее значение целевой переменной по датасету);
leave-one-out (рассчитывается среднее значение целевой переменной по всем объектам кроме текущего);
leave-bucket-out (рассчитывается среднее значение целевой переменной на одной части данных и применяется для оставшихся других);

Однако все вышеперечисленные способы так или иначе приводят к утечке данных и переобучению, и данную проблему можно частично решить с применением mean target encoding на отложенной выборке, что работает заметно лучше, но такой способ не идеален так как остаётся меньше данных для обучения и вычисления статистик по категориальным признакам.

В CatBoost применяется модифицированный подход к mean target encoding, называемый Ordered target (CatBoost) encoding, который состоит из 3-х нижеперечисленных этапов.

Случайная перестановка

Образцы в наборе данных перемешиваются случайным образом, что позволяет создать более сбалансированный набор данных и снизить переобучение.

Статистики «по прошлому»

После случайной перестановки рассчитывается новое значение для текущего категориального признака следующим образом:

$CTR = \frac{TargetSum + Prior}{TotalCount + 1}$

Prior — константа, определяемая как среднее арифметическое целевых значений.

TargetSum — сумма целевых значений для конкретного категориального признака до текущего момента.

TotalCount — количество всех наблюдений для конкретного категориального признака до текущего момента.

Данная формула может быть применима как для регрессии, так и для мультиклассификации.

Пошаговый пример с нуля для мультиклассификации

!pip install category_encoders

from category_encoders.cat_boost import CatBoostEncoder

train = pd.DataFrame({"color": ["red", "blue", "black", "black", "blue", "blue", "red", "blue"]})
target = pd.Series([2, 1, 2, 1, 0, 3, 0, 1])

cbe_encoder = CatBoostEncoder()
train_cbe = cbe_encoder.fit_transform(train, target)
prior = target.mean()

print(f'prior: {prior}', '', sep='\n')
print(train_cbe)


prior: 1.25

    color
0  1.2500
1  1.2500
2  1.2500
3  1.6250
4  1.1250
5  0.7500
6  1.6250
7  1.3125

В случае c числовыми признаками создаётся набор меток с помощью квантования и производится их бинаризация на k классов: все значения до границы (метки) относятся к классу 0, а после к классу 1. Далее процедура схожа со случаем выше.

В CatBoost поддерживаются следующие виды квантования:

мода;
медианная сетка;
равномерная сетка + квантили (размер квантования уменьшается вдвое и разбиения (пороговые значения) объединяются по одному из двух способов: медиана или равномерная сетка);
MaxLogSum — максимизируется значение следующего выражения внутри каждого сегмента:

$\sum\limits_{i=1}^{n} log \ w_i$

MinEntropy — минимизируется значение следующего выражения внутри каждого сегмента:

$\sum\limits_{i=1}^{n} w_i \ log \ w_i$

GreedyLogSum — максимизируется аппроксимация жадным способом следующего выражения внутри каждого сегмента:

$\sum\limits_{i=1}^{n} log \ w_i$

где n — число различных объектов в сегменте, w (weight) — число повторений объекта в сегменте.

Комбинация категориальных признаков

В CatBoost используется комбинация категориальных признаков (например, красный диван, синий диван, синий стул и так далее), что позволяет модели учитывать неочевидные взаимосвязи между ними и улучшить качество итоговых прогнозов в целом. Для избегания экспоненциального роста комбинаций категориальных признаков, их создание происходит не перед началом обучения, а в момент построения каждого дерева жадным алгоритмом отдельно.

Структура решающих деревьев и упорядоченный бустинг

В качестве базовых моделей в CatBoost используются симметричные односторонние деревья, имеющие одинаковую глубину и ширину. Создатели алгоритма считают, что такой подход позволяет ускорить обучение и создавать деревья, которые менее подвержены переобучению.

Для нахождения наилучшего критерия разбиения в узле используется косинусное сходство — его максимальное значение будет соответствовать лучшему критерию:

$Cosine \ Similarity = \frac{ \sum\limits_{i=1}^{n} A_i B_i}{\sqrt{\sum\limits_{i=1}^{n} A_i^2} \sqrt{\sum\limits_{i=1}^{n} B_i^2}}$

где A — вектор с остаточными ошибками, B — вектор с прогнозами в листовых узлах.

Проблема классического бустинга заключается в том, что прогнозы базовых моделей могут иметь высокое смещение, что в конечном счёте приводит к переобучению модели в целом — это обусловлено тем, что обучение и оценка деревьев производится на одинаковом наборе данных (X_train). Данная проблема решается с помощью метода линейной аппроксимации упорядоченного бустинга (Ordered Boosting), суть которого заключается в следующем: данные перемешиваются случайным образом и после чего текущее дерево обучается на m-1 образцах с учётом остатков предыдущего, что очень похоже на принцип в ordered target encoding, то есть изначально берётся один образец из набора данных и модель учится на всех образцах до него, то есть на нулях, далее берётся два образца и модель учится на одном, после чего рассчитываются остатки, которые будут использованы для следующей модели и так далее. Такой подход позволяет произвести обучение дерева со сложностью O(log(m)) вместо O(m^2) .

Структура симметричного одностороннего дерева

Дополнительные возможности CatBoost

Работа с пропусками

Поддерживаются следующие режимы обработки пропусков:

"Forbidden" — пропуски интерпретируются как ошибки.
"Min" — пропуски обрабатываются как минимальное значение (меньшее, чем все остальные значения) для объекта.
"Max" — пропуски обрабатываются как максимальное значение (превышающее все остальные значения) для объекта.

Режим обработки по умолчанию — Min. Также при выборе деревьев учитывается разбиение, отделяющее пропущенные значения от всех остальных.

Snapshots

Данный параметр (save_snapshot=True) позволяет создавать резервные копии промежуточных результатов. Если происходит неожиданное прерывание обучения (например, случайное выключение компьютера), то оно может быть продолжено из сохранённого состояния и в таком случае не нужно повторять завершённые итерации построения деревьев.

Обучение на GPU

В CatBoost возможно обучение не только на нескольких GPU, но и сразу на нескольких серверах с GPU, что фактически позволяет запустить распределённое обучение и использовать датасеты любого размера. Данная процедура позволяет значительно ускорить работу алгоритма: чем больше датасет, тем больше будет прирост в производительности в сравнении с обучением на CPU. Для того чтобы этим воспользоваться достаточно указать task_type="GPU".

Пакетная реализация CatBoost и оценка полученных результатов

Как можно заметить из примера классификации, не смотря на большое количество улучшений, CatBoost далеко не всегда показывает высокие результаты в сравнении с другими реализациями градиентного бустинга, как это заявляют его разработчики, особенно заметна разница в случае с несбалансированными наборами данных. Однако, стоит отметить, что в случае хорошей подготовки данных, при прочих равных, в ряде случаев CatBoost будет отличаться от других модификаций немного более высокой точностью.

В случае регрессии, когда нет явно выраженной проблемы как в случае выше, CatBoost справляется довольно хорошо.

Полученные результаты приведены ниже.

!pip install catboost

CatBoostClassifier

from catboost import CatBoostClassifier, CatBoostRegressor

cb_clf = CatBoostClassifier(random_seed=0, logging_level='Silent')
cb_clf.fit(X1_train, y1_train)
cb_clf_pred_res = cb_clf.predict(X1_test).squeeze()   # remove all single-dimensional entries
cb_clf_accuracy = accuracy_score(cb_clf_pred_res, y1_test)
print(f'cb_clf accuracy: {cb_clf_accuracy}')
print(cb_clf_pred_res)

feature_indexes = [1, 3]
title3 = 'CatBoostClassifier surface'
decision_boundary_plot(X1, y1, X1_train, y1_train, cb_clf, feature_indexes, title3)


cb_clf accuracy: 0.6851851851851852
[5 0 1 4 1 1 0 1 1 1 0 0 1 1 1 5 2 1 2 2 1 0 5 1 0 0 5 0 1 1 0 0 1 0 0 0 0
 0 0 5 1 5 1 0 1 1 0 1 0 1 0 1 5 0]

CatBoostRegressor

cb_reg = CatBoostRegressor(random_seed=0, logging_level='Silent')
cb_reg.fit(X2_train, y2_train)
cb_reg_pred_res = cb_reg.predict(X2_test)
cb_reg_mape = mean_absolute_percentage_error(cb_reg_pred_res, y2_test)
print(f'cb_reg mape: {cb_reg_mape}')
print(cb_reg_pred_res)


cb_reg mape: 0.3260573594870776
[268.21698587 226.61112865 152.56164473 105.39534082 185.56084388
 231.13386917 112.94031582 246.11217136 121.83631454 238.66069067
 193.92810529 171.53207065 120.95427566  98.87431921 301.0159263
  97.989809   170.68360243  70.03044356 104.9534818  235.1624283
 174.7895105  131.39173163 178.28683674 128.13631321 230.60611298
 186.37090017 131.52230822  61.453026   231.95028492 148.67901568
 186.2317954   80.45908918 137.68617262 171.24445717 128.77648933
 178.31364898 179.17236177 149.82094832 102.25676058 209.48231728
 116.01443961 169.06621701 110.9047463  210.24240477 158.28102362
  72.4532758  135.86931928 121.14865532  76.8103627  278.66222581
 127.85077736  55.25704176 144.51194393 170.06468442 271.51905928
 187.61879537 159.56680649 118.65520621 121.92004997 183.56316238
 248.87847858 137.77180529 117.98743897  82.53109543 255.59388802
 151.23062005  94.00606836 257.15481144 227.21705762  96.77322241
  78.34198817 118.09701594  91.2069645   91.70192792 125.65538331
 155.68222799 129.63411563 230.46496307 292.38275237 213.68057983
 131.76653094 217.93105223  50.37188156 256.16334997  94.4107044
  90.95547399 139.58356594 187.26517    127.97009716 171.85831366
  89.65051176  95.93129442  60.14437246 178.27466698 101.36657382
 106.15753799 206.11327567 217.74079838 130.83283251 138.63144499
 154.18156801 123.38315127 197.65233246 125.95249034 239.25024631
 139.69849648 245.02582928 288.17892899 122.76437963  70.023686
 233.36946028]

Преимущества и недостатки CatBoost

Преимущества:

высокая точность и скорость обучения в большинстве случаев;
возможность работы с категориальными признаками;
большое количество гиперпараметров для настройки алгоритма;
применяется для многих задач.

Недостатки:

нестабильная работа с несбалансированными данными;
сложность в конфигурации из-за большого количества гиперпараметров;
плохая документация (личное мнение).

Дополнительные источники

Статья «CatBoost: unbiased boosting with categorical features», Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin.

Документация.

Лекция.

Пошаговое построение CatBoost (в ordered target encoding есть ошибки):

один;
два.

Особенности LightGBM

LightGBM (Light Gradient Boosting Machine) — реализация градиентного бустинга от компании Microsoft, способная обрабатывать большие наборы данных и при этом обеспечивать высокую точность. Рассмотрим более подробно ключевые особенности, которые на это повлияли.

Gradient-based One-Side Sampling (GOSS)

В отличие от классического градиентного бустинга, где регрессионные деревья обучаются на всём множестве X_train, в LightGBM деревья обучаются на подмножестве данных, полученном с помощью одностороннего отбора на основе градиента (GOSS), суть которого заключается в сортировке и выборе образцов top N с наибольшим градиентом — это означает, что именно этим данным следует уделить особое внимание, поскольку на них дерево будет ошибаться больше всего. Помимо этого, случайным образом производится отбор части образцов с низким градиентом с последующим присвоением им весов $\frac{1-a}{b}$ , чтобы избежать изменений в распределении данных. В данном случае — доля образцов top N в процентах с наибольшим градиентом, — доля образцов в процентах с низким градиентом.

Проще говоря, использование GOSS позволяет значительно ускорить обучение каждого последующего дерева в ансамбле путём уменьшения обучающего набора данных без особой потери в качестве прогнозов в большинстве случаев.

Histogram-based tree node splitting

При поиске наилучшего критерия разбиения узла в LightGBM также можно использовать гистограмму значений признаков: перебираются все пороговые значения гистограммы вместо всех значений признаков, что позволяет существенно ускорить поиск наилучшего критерия. Например, для признака "рост" можно использовать вопросы "рост" ⩽ 160, "рост" ⩽ 170 вместо "рост" ⩽ 163, "рост" ⩽ 167 и так далее. Как можно было заметить, гистограммы применяются в том или ином виде во всех трёх модификациях градиентного бустинга.

Exclusive Feature Bundling (EFB)

Предположим, что тем или иным образом в датасете имеются закодированные категориальные признаки, большинство из которых являются взаимоисключающими (те, которые одновременно не принимают ненулевые значения). При большом числе таких признаков обучение модели может занять слишком много времени и для решения данной проблемы в LightGBM применяется понижение размерности пространства признаков путём объединения взаимоисключающих признаков (EFB).

Стоит отметить, что в действительности строго взаимоисключающие признаки в данных встречаются крайне редко. Поэтому вводится такое понятие как доля конфликтов между признаками — отношение наблюдений от общего числа, где два признака принимают одновременно ненулевые значения.

Допустим, имеется датасет с признаками F0−F4 . В первую очередь стоит установить допустимую долю конфликтов, например, 25%. Тогда в случае из 8 образцов допустимое число конфликтов будет составлять 2. Далее строится матрица конфликтов между всеми признаками и производится подсчёт общей суммы конфликтов для каждого признака, после чего происходит их сортировка в порядке убывания.

На основе матрицы конфликтов строится взвешенный граф, у которого в полученном порядке отсекаются все рёбра с числом конфликтов, превышающим допустимое значение.

Работа Exclusive Feature Bundling с нуля на простом примере

Первой связкой будет F_0−F_1 . Не смотря на то что F_1−F_2 и F_1−F_3 не конфликтуют, их объединение невозможно из-за конфликтов F_0−F_2 и F_0−F_3 . Следующей связкой будет F_2−F_4 . Не смотря на то, что F_4 и F_3 не конфликтуют, их объединение невозможно из-за конфликта F_2−F_3 . Последним остаётся признак F_3 .

Новая связка будет состоять из ненулевых значений первого признака и ненулевых значений второго плюс смещение, равное максимальному значению первого признака: смещение каждого следующего признака будет равно сумме максимальных значений предыдущих. В случаях, когда наблюдается конфликт, берётся значение последнего признака со смещением.

Подробные расчёты приведены ниже.

Нетрудно заметить, что алгоритм EFB можно оптимизировать, игнорируя вычисления для всех нулевых значений в признаках. Также данную оптимизацию можно применить к histogram-based tree node splitting, что в целом снизит сложность построения гистограммы с O(data) до O(non-zero data).

Leaf-wise tree growth

Вместо построения деревьев слой за слоем, в LightGBM деревья строятся в глубину: рост продолжается в листовом узле с наибольшим информационным приростом, в результате чего дерево получается глубоким и несимметричным. Чаще всего такой подход превосходит симметричные деревья в плане точности.

Обучение на GPU

При работе с большими датасетами в LightGBM поддерживается обучение не только на GPU, но и распределённое обучение сразу на нескольких машинах.

На данный момент существуют следующие подходы:

Data parallel (образцы датасета разбиваются на локальные гистограммы, каждая из которых обрабатывается отдельно);
Feature parallel (признаки датасета разбиваются на локальные гистограммы, каждая из которых обрабатывается отдельно);
Voting parallel (на каждом процессе локально выбирается K лучших признаков для разбиения, после чего выбирается наилучший из них глобально).

Пакетная реализация LightGBM и оценка полученных результатов

Как можно заметить, в случае классификации LightGBM показал результат из коробки значительно лучше, чем предыдущие модификации бустинга. Также стоит отметить, что из всех трёх рассмотренных модификацией градиентного бустинга LightGBM является самой быстрой: чем больше датасет, тем сильнее будет заметна разница в плане скорости.

Полученные результаты приведены ниже.

LGBMClassifier

from lightgbm import LGBMClassifier, LGBMRegressor

lgb_clf = LGBMClassifier(random_state=0, verbose=-1)
lgb_clf.fit(X1_train, y1_train)
lgb_clf_pred_res = lgb_clf.predict(X1_test)
lgb_clf_accuracy = accuracy_score(lgb_clf_pred_res, y1_test)
print(f'lgb_clf accuracy: {lgb_clf_accuracy}')
print(lgb_clf_pred_res)

feature_indexes = [1, 3]
title4 = 'LGBMClassifier surface'
decision_boundary_plot(X1, y1, X1_train, y1_train, lgb_clf, feature_indexes, title4)


lgb_clf accuracy: 0.7407407407407407
[5 0 1 4 1 1 0 1 1 1 0 0 0 1 1 5 2 1 2 2 3 1 5 5 0 0 5 0 1 1 0 0 1 0 0 0 0
 0 0 5 1 4 1 0 0 1 0 1 0 1 0 1 5 0]

LGBMRegressor

lgb_reg = LGBMRegressor(random_state=0, verbose=-1)
lgb_reg.fit(X2_train, y2_train)
lgb_reg_pred_res = lgb_reg.predict(X2_test)
lgb_reg_mape = mean_absolute_percentage_error(lgb_reg_pred_res, y2_test)
print(f'lgb_reg_mape: {lgb_reg_mape}')
print(lgb_reg_pred_res)


lgb_reg_mape: 0.349719771087187
[261.46864967 223.24674049 179.60231224 104.76984808 197.67529058
 264.26116883 111.11355079 243.72059654 144.18045853 239.08131542
 199.90841705 170.37417433 151.67933516  89.30779664 297.93561211
  96.58862229 162.53285879  70.06113305 108.36436227 234.16276627
 207.7552967  136.83913826 172.38724529 110.26981866 228.1574526
 163.56817258 108.94516511  73.93303203 233.31869769 116.00042252
 195.39846082  72.54403358 164.38943926 194.42418913 130.42154381
 183.04435501 173.56352142 147.00774306  89.07365731 175.25759644
 108.72749406 144.48000593  99.60713604 196.86889512 160.47835096
  60.17798032 133.4909559  115.46543242  64.48286456 251.20520649
 135.52608509  45.16396653 161.70420785 187.28406745 274.21158331
 179.30355071 150.1563969  111.1275583  100.5067047  133.7868607
 269.98464504 159.03980859 118.01578638  75.41625416 249.0569831
 133.99026562 111.43954756 221.81954307 206.02676942  81.56098377
  72.5920367  101.31664062  93.31261471 111.24487763 149.40048547
 177.37205369 130.08118649 212.0573     294.138509   232.24330644
 109.5457663  181.3487432   63.20386576 261.47508264  88.43007908
  99.59568927 127.97978786 211.77672798 121.44351586 138.27282283
 121.01455597  91.26965383  68.92870597 156.03234199 101.80734511
 108.60244399 214.13968395 218.58499151 131.09438025 114.99269899
 187.63688894 117.5399017  165.85959666 110.74981366 238.97897095
 114.01806217 240.07138204 283.0195311  108.08395213  67.44250127
 238.97271264]

Преимущества и недостатки LightGBM

Преимущества:

высокая точность прогнозов и особенно скорость обучения;
возможность работы с категориальными признаками;
большое количество гиперпараметров для настройки алгоритма;
применяется для многих задач.

Недостатки:

возможна нестабильная работа на датасетах небольшого размера;
сложность в конфигурации из-за большого количества гиперпараметров.

Дополнительные источники

Статья «LightGBM: A Highly Efficient Gradient Boosting Decision Tree», Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, Tie-Yan Liu.

Документация.

Лекции:

Какой бустинг лучше

Однозначного ответа на вопрос какой бустинг лучше нет. Если раньше можно было сказать, что при работе с категориальными признаками лучше использовать CatBoost, когда важна скорость — LightGBM, ну или XGBoost, если хочется стабильное решение и нет времени на эксперименты, то сейчас такое утверждение будет неверным: как можно было заметить, все три модификации успели скопировать друг у друга много хороших идей, но исторически они отличались довольно сильно.

Более правильным будет утверждение, что выбор наилучшего бустинга будет зависеть от конкретной задачи и данных, которые применяются для обучения модели: в разных сценариях одни и те же алгоритмы могут показывать как лучшие результаты в плане скорости и точности, так и наоборот — худшие. Таким образом, выбор наиболее оптимального бустинга будет заключаться в проведении экспериментов с разными модификациями и сравнении полученных результатов.

Несколько полезных источников со сравнением бустингов: один, два, три.

Стекинг и блендинг в ML 🡆

Источник

Теги

Категория

Новости

Дата

13 мар. 2024 г.

22.06.26 21:51 kimberlyhebert786

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
24.06.26 01:25 Fraddy Pual

I never thought it would happen to me—but I lost $256,100 in Bitcoin through a shady investment deal. I was shattered, panicked, and convinced my savings were gone forever. Just when I was about to give up, I stumbled upon reviews for FUNDSRETRIEVER, a cyber recovery team with a solid reputation. I decided to give it a shot, and to my absolute shock, they recovered every single cent in record time. Working with them was a lifesaver. If you've been tricked by fake investment platforms, don't lose hope—FUNDSRETRIEVER can help. Contact them here: Email: [email protected] | WhatsApp: +1603512144 8| Telegram: @Fundsretriever
24.06.26 01:27 Fraddy Pual

I never thought it would happen to me—but I lost $256,100 in Bitcoin through a shady investment deal. I was shattered, panicked, and convinced my savings were gone forever. Just when I was about to give up, I stumbled upon reviews for FUNDSRETRIEVER, a cyber recovery team with a solid reputation. I decided to give it a shot, and to my absolute shock, they recovered every single cent in record time. Working with them was a lifesaver. If you've been tricked by fake investment platforms, don't lose hope—FUNDSRETRIEVER can help. Contact them here: Email: [email protected] | WhatsApp: +16035121448 | Telegram: @Fundsretriever
24.06.26 01:28 Fraddy Pual

I never thought it would happen to me—but I lost $256,100 in Bitcoin through a shady investment deal. I was shattered, panicked, and convinced my savings were gone forever. Just when I was about to give up, I stumbled upon reviews for FUNDSRETRIEVER, a cyber recovery team with a solid reputation. I decided to give it a shot, and to my absolute shock, they recovered every single cent in record time. Working with them was a lifesaver. If you've been tricked by fake investment platforms, don't lose hope—FUNDSRETRIEVER can help. Contact them here: Email: [email protected] | WhatsApp: +16035121448 | Telegram: @Fundsretriever.
24.06.26 01:58 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.06.26 01:58 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.06.26 14:16 Universina da Mota

Becoming a victim of an investment scam is never anyone's intention it often happens because fraudsters exploit trust and a lack of awareness. I would like to express my sincere gratitude to the dedicated team at ResQpro for their professionalism and commitment to helping victims of online investment fraud. Their efforts in assisting individuals with the recovery of stolen assets and holding scammers accountable are truly commendable. If you need assistance or would like to learn more, you can contact them through: Email: [email protected] Alternative Email: [email protected] Telegram: @ResQprofirm WhatsApp: +1 (985) 296-9146
24.06.26 14:21 Elizabeth Thompson

If you believe you have been the victim of an investment scam, it is important to act promptly and gather all relevant information. Keep records of transaction receipts, wallet addresses, communication logs, account details, and any other evidence related to the incident. Providing accurate documentation can help investigators, financial institutions, legal professionals, or recovery specialists review your case and determine what options may be available. Be cautious of anyone who guarantees the recovery of lost funds or requests large upfront payments. For additional information, you may contact: Email: [email protected] Telegram: @ResQprofirm WhatsApp: +1 (985) 296-9146
24.06.26 15:33 Júlia Castro

If you have fallen victim to an investment scam, it is important to act quickly and gather all available evidence related to the incident. This may include transaction records, wallet addresses, screenshots of conversations, emails, account details, and any information connected to the individuals or entities involved. Having complete documentation can help professionals assess your situation and explore possible recovery options. Always exercise caution when seeking assistance and carefully verify the credentials of any service provider before proceeding. For further information, you may contact: Email: [email protected] Telegram: @ResQprofirm WhatsApp: +1 (985) 296-9146
24.06.26 22:01 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
24.06.26 22:01 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
25.06.26 21:13 Emilie Safi

A fraudulent investment scheme operated by BTCMining.limited functions as a fake return scam. In this setup, scammers lure victims with false promises of high returns. Through manipulative tactics, they gain individuals' trust and convince them to invest, ultimately leading to financial loss. If you have ever faced a cyber threat or fallen victim to an online crypto scam and need to reach the authorities, I recommend contacting [email protected], [email protected], WhatsApp +19852969146, telegram @resqprofirm. They are a legitimate team that helps victims of online crypto scams using advanced tools.
25.06.26 21:25 Emilie Safi

So I ended up losing $38,000 to this platform. At first, they kept asking me to put in more money so I could get into my portfolio. I did that, but then they wouldn’t let me withdraw anything—just kept asking for more deposits. It got way too suspicious, so I stopped. I found this company called ResQProfirm on Google and told them what happened. They got in touch, asked me to walk them through everything, and I gave them all the proof I had. They did an amazing job tracking down my money and getting it back. Big thanks to them at [email protected] and on WhatsApp at +19852969146. Please be careful out there and always research before investing.
26.06.26 01:04 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
26.06.26 01:04 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
26.06.26 02:48 Miriam Rocha

I trusted this platform with $120,000 of my hard-earned money. Then they started asking for more deposits just so I could access my own portfolio. I paid, but every withdrawal request was denied. They kept pushing for more money. I finally stopped it just felt wrong. Desperate, I found ResQProfirm on Google. They didn't just hear me out; they truly listened. I shared all my proof, and they launched an investigation. Thanks to their hard work, they tracked and returned my funds. From the bottom of my heart, thank you to [email protected] and their WhatsApp +19852969146. Please stay safe and always verify a platform before investing
26.06.26 02:52 Miško Bakić

I got my $232,000 refund thanks to [email protected] and WhatsApp +19852969146. Highly recommended for anyone in a similar situation.
26.06.26 02:56 Asunción Herrera

A recovery of $48,330 was facilitated by [email protected]. Individuals who have experienced financial fraud may consider contacting this service.
26.06.26 15:05 Riley Stephens

If withdrawals keep getting denied, stay calm. I went through the same, and this firm helped me recover everything. Their assistance was outstanding. Contact: [ResQProFirm @Gmail|•|com], Telegram: ResQprofirm, WhatsApp: <+198> <5296> <9146>.
26.06.26 15:09 Antonio Riley

Withdrawal troubles shouldn’t stress you out. I faced a similar problem, and this firm stepped in and recovered my funds. Their support truly mattered. Contact them: [[email protected], ResQprofirm @aol.com], Telegram: ResQprofirm, WhatsApp: +19852969146.
28.06.26 00:37 kimberlyhebertt673

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
28.06.26 00:37 kimberlyhebertt673

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
29.06.26 11:57 Lisadonato0726

For 43 years, I struggled with bad credit due to my own poor decisions, and my credit score was around 490. When my girlfriend and I decided to buy a house, a mortgage broker informed us that it would be impossible to secure a mortgage with my credit score. As a result, she referred me to a company called HACK MAVENS CREDIT SPECIALIST, assuring me of their professionalism and ability to assist with credit improvement. Upon contacting them, I was impressed by their professionalism and they assured me that they could help. In less than 6 days, my credit score skyrocketed to 785, and they also successfully resolved issues in my credit report, including the bankruptcy. I am incredibly satisfied with their service and would highly recommend HACK MAVENS CREDIT SPECIALIST for reliable credit repairs. You can reach them at H A C K M A V E N S 5 [AT] G M A I L [DOT] COM or at [+] [1] [2 0 9] [4 1 7] – [1 9 5 7]. Thanks to their help, my girlfriend and I are now proud homeowners.
29.06.26 22:37 riley777

Back in 2025, I watched my life savings vanish. A thief took every cent. I felt desperate and went looking for a way to get it back. I found a guy here who said he was an expert haha. He talked about special software that could find my missing cash. I trusted him. That was a big mistake. He was just another scammer. I paid him a software fee and then he just stopped answering my texts and ran off with my money too. I felt so ashamed that I kept quiet about it for months. It is hard to admit you got fooled twice. Later on, I found a real pro. she did not use a fancy sales pitch. she just looked at the trans screenshots and followed the path the money took. she worked fast and got my funds back into my account. Having that money back changed everything. I can sleep again. her info; [email protected]. Call/chatroom on Whtasapp/ +44 7476618364.
30.06.26 15:08 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
30.06.26 15:08 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
02.07.26 01:22 Lieneke Bonnema

I highly recommend ResQprofirm for their professional asset recovery services of my $120,000 scammed funds. Their expertise, professionalism, and commitment to achieving results make them a reliable choice for anyone seeking dependable recovery assistance. [email protected], WhatsApp +19852969146, telegram @resqprofirm
02.07.26 01:26 Clara Morin

I want to extend my deepest appreciation for showing that circumstances do not define one’s potential for greatness. Your support has been a major source of inspiration during my trading journey, and I am sincerely grateful for your insight and mentorship. Thank you so much. [email protected], WhatsApp +19852969146, telegram ResQprofirm
02.07.26 01:31 Robin Hale

I sincerely want to thank you for demonstrating that anyone can rise above their circumstances and achieve success. Your constant support has been incredibly inspiring during my trading journey, and your wisdom and advice mean so much to me. I appreciate you deeply. [email protected], WhatsApp +19852969146, telegram Resqprofirm
04.07.26 15:32 Fraddy Pual

There are few companies I trust as much as FUNDSRETRIEVER. When I lost $653,000 in Ethereum to a ruthless scam, I thought my life would never be the same. The betrayal cut deep, but I refused to give up. I searched tirelessly for a legitimate way to recover what was stolen, and finally found FUNDSRETRIEVER—the most competent and compassionate recovery team I could have imagined. They handled my case with precision and care, and in the end, my entire ETH wallet was restored. More than the money, they gave me back my hope and happiness. I'm sharing my story because I want others to know that recovery is possible. If a scam has taken from you, don't hesitate—contact FUNDSRETRIEVER today. Email: FUNDSRETRIEVER1@ Gmail.com | WhatsApp: +1 603-512-1448 | Telegram: @FUNDSRETRIEVER
05.07.26 14:44 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
05.07.26 14:44 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
06.07.26 16:20 Olga Ognjanović

Having trouble withdrawing funds from an investment platform? ResQprofirm provides fund recovery assistance for individuals seeking help with investment-related disputes. I reached out to them after experiencing problems with an investment platform, and I appreciated their professionalism and support throughout the process. If you're facing a similar situation, act promptly, keep records of your transactions and communications, and seek assistance from a qualified recovery service or the appropriate authorities. Contact: Email: [email protected] Telegram: @ResQprofirm WhatsApp: +1 985 296 9146
06.07.26 16:31 Joseph Weigl

Invest wisely and stay cautious. Don't be influenced by promises of unusually high returns or convincing sales pitches from brokers. I learned this the hard way after falling victim to an investment scam that promised huge profits. Fortunately, I acted quickly and reported the incident to a recovery firm for assistance. Contact: Email: [email protected] Telegram: @Resqprofirm WhatsApp: +1 985 296 9146
06.07.26 16:33 Jaran Løvlien

A heartfelt thank you to RESQPRO FIRM for their commitment and professionalism throughout the investigation of my case. Their team worked diligently and helped recover assets valued at $88,000, which were returned to my wallet. I truly appreciate their support, clear communication, and dedication, and I'm grateful for the assistance I received. Contact: Email: [email protected] Telegram: @Resqprofirm WhatsApp: +1 985 296 9146
07.07.26 18:00 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
07.07.26 18:01 robertalfred175

CRYPTO SCAM RECOVERY SUCCESSFUL – A TESTIMONIAL OF LOST PASSWORD TO YOUR DIGITAL WALLET BACK. My name is Robert Alfred, Am from Australia. I’m sharing my experience in the hope that it helps others who have been victims of crypto scams. A few months ago, I fell victim to a fraudulent crypto investment scheme linked to a broker company. I had invested heavily during a time when Bitcoin prices were rising, thinking it was a good opportunity. Unfortunately, I was scammed out of $120,000 AUD and the broker denied me access to my digital wallet and assets. It was a devastating experience that caused many sleepless nights. Crypto scams are increasingly common and often involve fake trading platforms, phishing attacks, and misleading investment opportunities. In my desperation, a friend from the crypto community recommended Capital Crypto Recovery Service, known for helping victims recover lost or stolen funds. After doing some research and reading multiple positive reviews, I reached out to Capital Crypto Recovery. I provided all the necessary information—wallet addresses, transaction history, and communication logs. Their expert team responded immediately and began investigating. Using advanced blockchain tracking techniques, they were able to trace the stolen Dogecoin, identify the scammer’s wallet, and coordinate with relevant authorities to freeze the funds before they could be moved. Incredibly, within 24 hours, Capital Crypto Recovery successfully recovered the majority of my stolen crypto assets. I was beyond relieved and truly grateful. Their professionalism, transparency, and constant communication throughout the process gave me hope during a very difficult time. If you’ve been a victim of a crypto scam, I highly recommend them with full confidence contacting: Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text: +1 (336) 390-6684 Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
09.07.26 19:06 Toivo Walli

I lost 8.56btc to a fake Bitcoin mining site, I tried withdrawing but couldn't approved my process, I reported to !R£SQPROFIRM! via °R£SQproFirm°àt°gmail•com° °tEL£°gram=R£SQprofirm °whaT°Zap+198°52°96°91°46
09.07.26 19:10 Misty Alexander

Ongoing messages demanding more money before approving withdrawals are a major red flag. Stop engaging and report the incident to a trusted recovery team. For professional support, you can contact R£sQprofirm using °ResQproFirm°àt°g,*ma'il(•)¢m°, TEL£gram ResQprofirm, or |whaTZap| +1-985-296-9146.
09.07.26 19:13 Clara Soto

Anyone receiving continued requests for additional deposits from a scam platform should immediately cut off communication and submit the case to a reputable recovery service for investigation. R£sQprofirm is a dependable firm you can reach at °ResQproFirm°àt°g,*ma'il(•)¢om°, TEL£gram ResQprofirm, or |whaTZap| +1-985-296-9146.
12.07.26 03:30 Kora Baltacha

Time is critical. Act now by reaching out to a reputable, seasoned recovery specialist who will guide you every step of the way. You'll need to submit transaction proof, scammer details, and any other useful information. Armed with this, the experts can trace and attempt to pull your money back from the scammers' hidden accounts or wallets. Best of all, R£sQprofirm provides recovery help without charging any upfront fees. Contact them immediately via Telegram @ResQprofirm, WhatsApp +19852969146, or email [email protected].
12.07.26 03:33 Pahal Mathew

It's important to move proactively by engaging an experienced recovery specialist. They will assist you throughout the process. To help them, provide: · Transaction evidence · Scammer information · Any additional relevant details The experts will then track and try to retrieve your funds from the scammers' hidden accounts or wallets. R£sQprofirm offers recovery assistance with no upfront fees. Contact: Telegram: @ResQprofirm WhatsApp: +19852969146 Email: [email protected]
13.07.26 23:49 [email protected]

One of the biggest concerns I have about cryptocurrency is the lack of regulation. It creates opportunities for scammers to invent convincing stories and fraudulent investment schemes. Unfortunately, some social media platforms continue to display these ads because they profit from them, even after users report them.I personally clicked on a Facebook advertisement for a company called Chickenfastmining and ended up losing more than $120,000 in a scam. I reported the ad, but nothing was done. Later, through a Reddit community, I found a recovery service called CYBERBERSPY that, in my personal experience, they helped me recover $110,000 of my lost funds. If you've been a victim of a cryptocurrency scam, don't lose hope. Explore your options carefully, and always verify the legitimacy of any recovery service before trusting them or paying any fees. Based on my own experience, CYBERBERSPY was helpful to me and i was able to recover my funds back, but I encourage everyone to do their own research before using any recovery service.i highly recommend: ([email protected])
15.07.26 11:53 Sarah Green

Thank you for showing that success is possible regardless of where someone starts. Your encouragement, valuable advice, and continuous support have inspired me throughout my $160,457k crypto investment recovery journey. I truly appreciate your kindness and dedication. Resqprofirm @gmail.com Telegram: Resqprofirm
15.07.26 11:58 Lily Gagné

I sincerely appreciate you for proving that anyone can overcome challenges and achieve success. Your unwavering support throughout my trading investment scam of $88,890 recovery journey has been truly inspiring, and your guidance and wisdom have meant a great deal to me. Thank you for everything ResQprofirm@ gmail.com, ResQprofirm on the telegram.
16.07.26 21:38 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
16.07.26 21:38 patricialovick86

How To Recover Your Bitcoin Without Falling Victim To Scams: A Testimony Experience With Capital Crypto Recover Services, Contact Telegram: @Capitalcryptorecover Dear Everyone, I would like to take a moment to share my positive experience with Capital Crypto Recover Services. Initially, I was unsure if it would be possible to recover my stolen bitcoins. However, with their expertise and professionalism, I was able to fully recover my funds. Unfortunately, many individuals fall victim to scams in the cryptocurrency space, especially those involving fraudulent investment platforms. However, I advise caution, as not all recovery services are legitimate. I personally lost $273,000 worth of Bitcoin from my Binance account due to a deceptive platform. If you have suffered a similar loss, you may be considering crypto recovery, The Capital Crypto Recover is the most knowledgeable and effective Capital Crypto Recovery Services assisted me in recovering my stolen funds within 24 hours, after getting access to my wallet. Their service was not only prompt but also highly professional and effective, and many recovery services may not be trustworthy. Therefore, I highly recommend Capital Crypto Recover to you. i do always research and see reviews about their service, For assistance finding your misplaced cryptocurrency, get in touch with them, They do their jobs quickly and excellently, Stay safe and vigilant in the crypto world. Contact: [email protected] You can reach them via email at [email protected] OR Call/Text Number +1 (336)390-6684 his contact website: https://recovercapital.wixsite.com/capital-crypto-rec-1
17.07.26 19:24 laimqq90

I recommend Marie when it comes to recovering lost/stolen ust/bitcoin or any kind of cryptocurrencies' from fake investment platforms because they're well specialized in that area and you'll get your money back in full. I can boldly say this right now based on my prior deal i had with them; she was the only one who was able to recover my lost money $52,760 dollars back to my account, Only * ([email protected] and WhatsApp +1 7127594675 successful in recovering my money. They are the only one who can fully restore your lost funds to your account without any deductions, I really value their work and am recommending her to you today. THANK ME LATER
17.07.26 20:12 martinsjude080

Needs Online Fraud Help Contact Mighty Hacker Recovery https://mightyhackarrecovery.com I lost $292,900 in Bitcoin after investing with an online mining company. After realizing I had been scammed, I spent a long time searching for ways to recover my funds and contacted several services without success. During my research, I came across Mighty Hacker Recovery through Google and YouTube. What caught my attention was that they said they would not require any upfront payment before providing their recovery service. I decided to contact them to discuss my case and understand their process. Throughout the process, they kept me informed about the progress. After several days, I was asked to provide my Bitcoin wallet address, and my case was concluded. Their fee was handled after the service rather than being requested in advance. If you've been the victim of a cryptocurrency scam, it's important to do your own research, ask questions, and carefully evaluate any recovery service before proceeding. Every case is different, so take the time to verify information and understand the process before making any decisions. For Bitcoin scam recovery, cryptocurrency scam, crypto recovery service, recover stolen Bitcoin, Bitcoin fraud help, blockchain investigation, crypto wallet recovery, online investment scam, digital asset recovery, and crypto scam support. Contact Them on WhatsApp +1 (343) 947-3496 or [email protected] or [email protected] or https://mightyhackarrecovery.com Scam recovery Online fraud help Scam alert Report fraud Fraud investigation Fake website checker Scam checker Identity theft protection Consumer protection Chargeback for scam Wire transfer scam Tech support scam Employment scam Rental scam Shopping scam Email scam WhatsApp scam Telegram scam Facebook scam Instagram scam
18.07.26 17:12 Malthe Larsen

My experience with OKX has been deeply frustrating. For months, my account withdrawals were restricted with no explanation or resolution. Multiple emails to their support team were ignored, leaving me without answers or reassurance. This complete lack of communication shattered my trust. I ultimately regained access to my funds only through the help of a third-party recovery service, ResQProfirm. What should have been a reliable platform instead made me feel helpless and cut off from my own assets. [email protected], WhatsApp +19852969146, telegram @Resqprofirm
18.07.26 17:42 پریا رضایی

OKX has been a major disappointment. My withdrawals were restricted for months, and repeated attempts to contact support went unanswered. This silence destroyed my confidence in the platform. I was only able to recover my funds through a third-party service, ResQProfirm. I trusted OKX for its reliability, but the experience left me feeling trapped and helpless. Timely communication and access to one’s own money should be a basic standard. [email protected], WhatsApp +19852969146, telegram: ResQprofirm
18.07.26 17:46 Adem Akışık

I truly didn’t expect such an outstanding outcome. Recovering my $49,360 felt impossible at first, but ResQprofirm’s dedication and persistence made it happen. I hold their team in the highest regard. [email protected], WhatsApp +19852969146
18.07.26 23:51 bernalzenaida

WhatsApp https://wa.link/fhle97 Telegram https://msng.link/o?@techcyberforc=tg As cryptocurrency continues to reshape global finance, cybercriminals are finding new ways to exploit investors through scams, hacks, phishing attacks, fake investment platforms, and other forms of digital asset fraud. For many victims, knowing where to turn after a loss can be one of the biggest challenges. Techy Force Cyber Retrieval was founded with one clear mission: to give victims of crypto fraud a fighting chance through professional blockchain investigations and cybersecurity expertise. Our team brings together experienced blockchain analysts, digital forensic specialists, cybersecurity professionals, and legal partners who work collaboratively to investigate cryptocurrency-related crimes. Using advanced blockchain forensic tools and global investigative techniques, we analyze transaction histories, trace digital asset movements where possible, identify valuable investigative leads, and prepare evidence that may assist clients and the appropriate authorities. We believe blockchain should represent transparency, accountability, and trust—not fear. That’s why we’re committed to helping victims understand their options, navigate the investigative process, and take informed action after cryptocurrency fraud. Every case is different, and while no legitimate recovery service can promise a successful recovery, acting quickly and working with experienced professionals can improve the quality of an investigation. At Techy Force Cyber Retrieval, we do more than investigate digital crimes—we advocate for victims, pursue the facts, and help people regain confidence after cryptocurrency fraud. WhatsApp https://wa.link/fhle97 Telegram https://msng.link/o?@techcyberforc=tg Crypto fraud doesn’t have to be the end of the road. It’s where our investigation begins.
19.07.26 04:10 Fraddy Pual

I can't thank Fundsretriever enough for everything they did for me. My name is Vanessa Conway, and I'm here to tell you my story of how I recovered money I never thought I'd see again. A few months ago, I put a significant amount of money into what looked like a genuine online investment company. At first, it felt real—they showed me fake profits and convinced me to invest even more. But when I tried to cash out, they went quiet and started asking for extra fees. That's when it hit me—I had been scammed. I was heartbroken, frustrated, and didn't know where to turn. That money was my savings—months of hard work gone. Then I found Fundsretriever online. I reached out, hoping for a miracle. Right away, their team made me feel heard. They were responsive, knowledgeable, and walked me through everything. They didn't just take my case—they took it seriously and kept me in the loop every step of the way. I finally felt like I had real experts fighting for me. And guess what? They actually got my money back. It wasn't instant, and it took teamwork, but it happened. When I saw those funds returned, I cried with joy. I'm sharing this so that anyone else out there who's been scammed knows—don't lose hope. Do your research before investing, and stay far away from platforms that promise too much too fast. And if you've already been stung, don't wait—get professional help immediately. Thank you, Fundsretriever, from the bottom of my heart. You didn't just recover my money—you restored my faith. — Vanessa Conway 📧 [email protected] 📱 WhatsApp: +16035121448 💬 Telegram: @Fundsretriever
19.07.26 19:53 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
19.07.26 19:53 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
19.07.26 19:54 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
30.07.26 00:04 Ahmed

A really cool analysis, thank you. I was especially struck by how strictly the order is defined: data processing and formatting come first, with color only at the very end. This really saves you from the typical mistake of "first a pretty palette, then we figure out what the chart is for." And the palette validator with OKLCH + colorblindness check is absolutely fantastic; you almost never see that in regular tools. By the way, while reading about rank-trajectory and the logarithmic scale, I immediately remembered how convenient it is to analyze dynamics on charts in ExpertOption—I've been trading there for quite some time now. When the data is well visualized, decisions are noticeably easier and more relaxed. I also liked the point about "no more than eight colors" and the dual-axis ban. Strict restrictions sometimes actually produce better results than complete freedom. I'll try this approach myself.
30.07.26 17:27 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
30.07.26 17:27 wendytaylor015

My name is Wendy Taylor, I'm from Los Angeles, i want to announce to you Viewer how Capital Crypto Recover help me to restore my Lost Bitcoin, I invested with a Crypto broker without proper research to know what I was hoarding my hard-earned money into scammers, i lost access to my crypto wallet or had your funds stolen? Don’t worry Capital Crypto Recover is here to help you recover your cryptocurrency with cutting-edge technical expertise, With years of experience in the crypto world, Capital Crypto Recover employs the best latest tools and ethical hacking techniques to help you recover lost assets, unlock hacked accounts, Whether it’s a forgotten password, Capital Crypto Recover has the expertise to help you get your crypto back. a security company service that has a 100% success rate in the recovery of crypto assets, i lost wallet and hacked accounts. I provided them the information they requested and they began their investigation. To my surprise, Capital Crypto Recover was able to trace and recover my crypto assets successfully within 24hours. Thank you for your service in helping me recover my $647,734 worth of crypto funds and I highly recommend their recovery services, they are reliable and a trusted company to any individuals looking to recover lost money. Contact email [email protected] OR Telegram @Capitalcryptorecover Call/Text Number +1 (336)390-6684 his contact: [email protected] His website: https://recovercapital.wixsite.com/capital-crypto-rec-1
31.07.26 16:21 rssllhrnsb

I learned an important lesson after investing in what appeared to be a genuine opportunity. Unfortunately, I was unable to access my funds, which reinforced the importance of carrying out thorough due diligence, checking whether an investment is appropriately regulated, and seeking qualified professional advice rather than relying solely on online reviews or testimonials. During the process of addressing my case, I worked with Mrs. Doris Ashley, who communicated clearly, provided regular updates, and handled the matter in a professional manner. According to my experience, I have recovered $50,000 so far, while efforts to resolve the remaining balance are still in progress. If you wish to contact her, the details I used are: Mrs. Tatiana Sorina TEXT : (tatianasorina06 at G.Ma IL ..c 0 m ) Before committing money to any investment, take time to verify the legitimacy of the platform, confirm any relevant regulatory authorisations, and avoid investing more than you can comfortably afford to lose.
01.08.26 15:05 keithwilson9899

ETHEREUM RECOVERY ASSISTANCE: CAPITAL CRYPTO RECOVER HELPED ME RECOVER $98,000 WORTH OF LOST ETH In cases of cryptocurrency scams, having accurate information and trusted support is essential. I would like to recommend Capital Crypto Recover Service, a professional team that specializes in assisting individuals with the recovery of lost or stolen Bitcoin and Ethereum (ETH). Their experienced experts are dedicated to helping victims of digital asset fraud by carefully analyzing each case, developing strategic recovery plans, Capital Crypto Recover Service knowledgeable team's primary goals are to satisfy clients and offer significant support and working diligently toward fund retrieval. The team is committed to providing reliable assistance and maintaining a high level of client satisfaction. Based on my assessment, their reputation professionalism and a strong commitment to their clients. If you have experienced a cryptocurrency loss, you can contacting them for further assistance Phone (Call/Text): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
01.08.26 15:05 keithwilson9899

ETHEREUM RECOVERY ASSISTANCE: CAPITAL CRYPTO RECOVER HELPED ME RECOVER $98,000 WORTH OF LOST ETH In cases of cryptocurrency scams, having accurate information and trusted support is essential. I would like to recommend Capital Crypto Recover Service, a professional team that specializes in assisting individuals with the recovery of lost or stolen Bitcoin and Ethereum (ETH). Their experienced experts are dedicated to helping victims of digital asset fraud by carefully analyzing each case, developing strategic recovery plans, Capital Crypto Recover Service knowledgeable team's primary goals are to satisfy clients and offer significant support and working diligently toward fund retrieval. The team is committed to providing reliable assistance and maintaining a high level of client satisfaction. Based on my assessment, their reputation professionalism and a strong commitment to their clients. If you have experienced a cryptocurrency loss, you can contacting them for further assistance Phone (Call/Text): +1 (336) 390-6684 Email: [email protected] Alternate Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
03.08.26 20:05 Philip

I was a victim of a crypto theft involving a Pink Drainer, which resulted in the theft of my Wrapped Bitcoin (WBTC) from my Polygon network. The experience was incredibly frustrating and distressing, as I had no idea how to recover my funds. Unfortunately, by the time I noticed the theft, the funds had already been drained to an address that I had no control over, making it seem like an irreversible situation.The attack happened when I clicked on what appeared to be a legitimate link. I didn’t realize at the time that it was a phishing attempt designed to siphon off my private keys and access my wallet. The Pink Drainer, a type of malicious script used by attackers, is specifically designed to exploit such vulnerabilities in crypto wallets. The moment I realized that my WBTC had been drained from my Polygon network. I felt completely helpless, as I didn’t have direct access to the thief's address, and there was no way to reverse the transaction on my own at that point, I started searching for ways to recover my funds, but most resources only offered generic advice that wasn’t practical in this particular case. I quickly realized that if I wanted to have any hope of getting my assets back, I would need professional assistance. After some research, I came across a reliable and trusted recovery team called Aspen Recovery Experts. Their expertise in cryptocurrency recovery, especially in cases like mine, seemed promising.I decided to reach out to Aspen Recovery Experts, and I’m incredibly grateful that I did. Their team of crypto recovery experts was able to help me trace the stolen funds and identify the path the funds took after they left my wallet. Using advanced tools and techniques, they were able to track the transactions on the blockchain, helping me understand where my WBTC had been sent. More importantly, they worked tirelessly to assist me in contacting the necessary parties and even interfaced with blockchain analysts to help facilitate the recovery process.Thanks to their efforts, I was able to successfully recover my stolen funds. The entire process took some time, but the Aspen Recovery Experts dedicated team provided regular updates and kept me informed throughout the process, which gave me a sense of hope and relief during an otherwise stressful time. If you’re ever in a similar situation, I highly recommend reaching out to a trusted recovery team like Aspen Recovery Expertshhh . Their professionalism, knowledge, and expertise were critical in helping me recover my funds and regain control of my crypto assets. Whatsapp : +1 747 231 9036 Telegram : @ prohackerspy Email : [email protected]
03.08.26 20:06 Philip

I was a victim of a crypto theft involving a Pink Drainer, which resulted in the theft of my Wrapped Bitcoin (WBTC) from my Polygon network. The experience was incredibly frustrating and distressing, as I had no idea how to recover my funds. Unfortunately, by the time I noticed the theft, the funds had already been drained to an address that I had no control over, making it seem like an irreversible situation.The attack happened when I clicked on what appeared to be a legitimate link. I didn’t realize at the time that it was a phishing attempt designed to siphon off my private keys and access my wallet. The Pink Drainer, a type of malicious script used by attackers, is specifically designed to exploit such vulnerabilities in crypto wallets. The moment I realized that my WBTC had been drained from my Polygon network. I felt completely helpless, as I didn’t have direct access to the thief's address, and there was no way to reverse the transaction on my own at that point, I started searching for ways to recover my funds, but most resources only offered generic advice that wasn’t practical in this particular case. I quickly realized that if I wanted to have any hope of getting my assets back, I would need professional assistance. After some research, I came across a reliable and trusted recovery team called Aspen Recovery Experts. Their expertise in cryptocurrency recovery, especially in cases like mine, seemed promising.I decided to reach out to Aspen Recovery Experts, and I’m incredibly grateful that I did. Their team of crypto recovery experts was able to help me trace the stolen funds and identify the path the funds took after they left my wallet. Using advanced tools and techniques, they were able to track the transactions on the blockchain, helping me understand where my WBTC had been sent. More importantly, they worked tirelessly to assist me in contacting the necessary parties and even interfaced with blockchain analysts to help facilitate the recovery process.Thanks to their efforts, I was able to successfully recover my stolen funds. The entire process took some time, but the Aspen Recovery Experts dedicated team provided regular updates and kept me informed throughout the process, which gave me a sense of hope and relief during an otherwise stressful time. If you’re ever in a similar situation, I highly recommend reaching out to a trusted recovery team like Aspen Recovery Expertshhh . Their professionalism, knowledge, and expertise were critical in helping me recover my funds and regain control of my crypto assets. Whatsapp : +1 747 231 9036 Telegram : @ prohackerspy Email : [email protected]
04.08.26 11:05 Kisnoles

Excellent analysis, thank you. I was particularly struck by how rigidly the skill sets the order: data processing and form come first, with color coming last. This really cures the habit of "first a pretty palette, then we'll figure it out." A palette validator using OKLCH + color blindness + WCAG is something most chart generators lack. And regarding the boundaries of competence, it's very clear. Where there's a self-checking loop (color), you delegate freely. Where there's a heuristic (form, data interpretation), you remain the final filter. This is a universal principle, not just for /dataviz. By the way, when you look at trading dashboards (including those of decent platforms like ExpertOption), it's immediately clear who thought about readability and who just threw in rainbow lines. Tools like this skill could greatly improve the quality of analytics. Thanks again for the detailed analysis – saved.
04.08.26 12:37 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
04.08.26 12:37 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] Call/Text Number: +1 (336) 390-6684
06.08.26 08:11 ROMMYHENDERSON344

All you need is to hire an expert to help you accomplish that. If there's any need to spy on your partner's phone. From my experience I lacked evidence to confront my husband on my suspicion on his infidelity, until I came across REALCYBERHACKERS which many commend him of assisting them in their spying mission. So I contacted him and he provided me with access into his phone to view all text messages, call logs, WhatsApp messages and even her location. This evidence helped me move him off my life . I recommend you consult REALCYBERHACKERS AT gmail com or whatsapp +14106350697 if you need access to your partner's phone or any kind of hacking, they carry out all kinds of hacking job
06.08.26 13:56 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
06.08.26 13:56 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/Text: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
11.08.26 03:43 raymont0714

I recommend a trusted cybersecurity PRO with experience in authorized device access, security testing, and data recovery. SHE work only with the owner's consent and follow all legal and privacy rules on meta data. With permission, this specialist can recover lost files, check device security offline & online, and review texts, call records, or hidden data (spy or lurk around a cheating partner or colleuge). Strict confidentiality agreements protect the information, and client details remain private. The work is careful, efficient, and suited to complex cases. For help securing or recovering information from a phone or another electronic device, use the details below 2 consult [email protected] +44 7476618364 I trust her secrecy a living witness. Her discretion is unmatched, ensuring your privacy is always maintained
12.08.26 16:37 rssllhrnsb

I learned an important lesson after investing in what appeared to be a genuine opportunity. Unfortunately, I was unable to access my funds, which reinforced the importance of carrying out thorough due diligence, checking whether an investment is appropriately regulated, and seeking qualified professional advice rather than relying solely on online reviews or testimonials. During the process of addressing my case, I worked with Mrs. Tatiana Sorina , who communicated clearly, provided regular updates, and handled the matter in a professional manner. According to my experience, I have recovered $50,000 so far, while efforts to resolve the remaining balance are still in progress. If you wish to contact her, the details I used are: Mrs. Tatiana Sorina TEXT : (tatianasorina06 at G_Ma IL dot ..c 0 m)…. Before committing money to any investment, take time to verify the legitimacy of the platform, confirm any relevant regulatory authorisations, and avoid investing more than you can comfortably afford to lose.
16.08.26 01:44 Matt Kegan

CapitalNode Analytics. They help to investigate and recover stolen digital assets from fake trading platforms. Great firm i must say.
18.08.26 04:59 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone WhatsApp/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
18.08.26 04:59 marcushenderson624

Bitcoin Recovery Testimonial After falling victim to a cryptocurrency scam group, I lost $354,000 worth of USDT. I thought all hope was lost from the experience of losing my hard-earned money to scammers. I was devastated and believed there was no way to recover my funds. Fortunately, I started searching for help to recover my stolen funds and I came across a lot of testimonials online about Capital Crypto Recovery, an agent who helps in recovery of lost bitcoin funds, I contacted Capital Crypto Recover Service, and with their expertise, they successfully traced and recovered my stolen assets. Their team was professional, kept me updated throughout the process, and demonstrated a deep understanding of blockchain transactions and recovery protocols. They are trusted and very reliable with a 100% successful rate record Recovery bitcoin, I’m grateful for their help and highly recommend their services to anyone seeking assistance with lost crypto. Contact: [email protected] Phone WhatsApp/Text Number: +1 (336) 390-6684 Email: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
19.08.26 14:50 BAYER7043

My account was locked, and I couldn’t access my own information until [email protected] +447476618364 whats?up? helped me recover it. This lady hacker was kind and professional, but this experience showed me how risky account lockouts can be. If you’re facing the same problem, do not panic consult her A. S. A. P. Be careful with recovery agents outchea, and research their name, business, and reviews before sharing any details. Never send passwords, security codes, banking information, or copies of your ID unless you’ve confirmed who you’re dealing with. Be wary of promises that sound too good to be true, especially claims that require little information or guarantee instant access with outrageous fees.
20.08.26 11:32 michaeldavenport218

I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR WhatsApp Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1
20.08.26 11:32 michaeldavenport218

I was recently scammed out of $53,000 by a fraudulent Bitcoin investment scheme, which added significant stress to my already difficult health issues, as I was also facing cancer surgery expenses. Desperate to recover my funds, I spent hours researching and consulting other victims, which led me to discover the excellent reputation of Capital Crypto Recover, I came across a Google post It was only after spending many hours researching and asking other victims for advice that I discovered Capital Crypto Recovery’s stellar reputation. I decided to contact them because of their successful recovery record and encouraging client testimonials. I had no idea that this would be the pivotal moment in my fight against cryptocurrency theft. Thanks to their expert team, I was able to recover my lost cryptocurrency back. The process was intricate, but Capital Crypto Recovery's commitment to utilizing the latest technology ensured a successful outcome. I highly recommend their services to anyone who has fallen victim to cryptocurrency fraud. For assistance contact [email protected] and on Telegram OR WhatsApp Number +1 (336)390-6684 via email: [email protected] you can visit his website: https://recovercapital.wixsite.com/capital-crypto-rec-1
23.08.26 20:02 leslieyee

Excellent analysis by /dataviz! I was particularly struck by how strictly the order is defined: "data meaning first, color last" and how the validator actually adjusts the palette according to OKLCH, color blindness, and WCAG standards. These rules greatly improve the quality of charts. By the way, a similar approach to clean and legible charts is highly valued in trading—for example, on the ExpertOption platform, the interface and price visualization are designed to ensure information is read instantly and without unnecessary noise.
25.08.26 13:44 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
25.08.26 13:44 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
30.08.26 15:37 [email protected]

Discovering I had been defrauded, I searched online for services claiming to recover stolen ETH. I contacted several companies, but none succeeded. Although some claimed they could trace assets or recover funds, I could not verify their success or recover my money. I later found SYLVESTER BRYANT Recovery through Google. After contacting him, he recovered 450,000,00. He can be reached at [email protected] or WhatsApp at +1 512 577 7957.
30.08.26 15:37 [email protected]

Discovering I had been defrauded, I searched online for services claiming to recover stolen ETH. I contacted several companies, but none succeeded. Although some claimed they could trace assets or recover funds, I could not verify their success or recover my money. I later found SYLVESTER BRYANT Recovery through Google. After contacting him, he recovered 450,000,00. He can be reached at [email protected] or WhatsApp at +1 512 577 7957.
01.09.26 11:16 lisawerth897

My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684
01.09.26 11:16 lisawerth897

My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684
01.09.26 17:34 Garry42

Really crazy world. These fraudsters go at any length to steal your hard earned funds. I have been a victim of a bitcoin scam about 7 months back. A Con artist gained access to my cashapp account through a phishing scam. They stole $409,000. I was really devastated. I did everything to get back my funds by contacting the FBI but they claimed there was nothing they could do. A friend told me about a recovery expert. He helps fight against various phishing and investment scams and they were able to help trace and recover my funds even though it took over 2 days. you can reach out to him through recoverydarek@gmail. com . I can guarantee his services are still active.
01.09.26 17:34 Garry42

Really crazy world. These fraudsters go at any length to steal your hard earned funds. I have been a victim of a bitcoin scam about 7 months back. A Con artist gained access to my cashapp account through a phishing scam. They stole $409,000. I was really devastated. I did everything to get back my funds by contacting the FBI but they claimed there was nothing they could do. A friend told me about a recovery expert. He helps fight against various phishing and investment scams and they were able to help trace and recover my funds even though it took over 2 days. you can reach out to him through recoverydarek@gmail. com . I can guarantee his services are still active.
02.09.26 03:43 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684
02.09.26 03:43 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684
04.09.26 21:51 Kovengray

Recovering your lost investment funds as the case might be, is not what you can do alone, you’d require the service of a trained recovery specialist. A recovery specialist is a person or a group of people who are well equipped to work around the brokerage network. They have vast knowledge about the whole network and have the right software and private keys to follow any transaction. I was ripped off trading online to an investment broker, good thing I got every penny back through the help of Gavin ray he’s a genius Contact : Gavinray78 at gmail com or WhatsApp +1 352 322 2096 It is also important to be patient and really calm during the process.
05.09.26 21:38 [email protected]

I invested 45,000 Euro, and later aggreviated to 198,000 Euro. I requested to place ‎Withdrawal of my funds to be paid to my Bank account. But nothing happened I was subjected to pay more fee until I can get my funds on my account, i got in touch with Theodore ryan here on this platform who had helped a lot of people, I followed all instructions and he legally got back my withheld funds, I got threatened by the company that if I don’t pay they will get my account frozen, all thanks to Theodore ryan and I highly recommend him to anyone dealing with an unregulated broker company… contact him on his Gmail - theodoreryan318@ gmail . com
09.09.26 21:31 lisawerth897

My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684
09.09.26 21:31 lisawerth897

My Experience With Cryptocurrency — CAPITAL CRYPTO RECOVER I lost over $82,000 in Bitcoin after falling victim to a fraudulent online investment scheme. After realizing I had been scammed, I spent considerable time researching possible ways to recover my funds. During my research, I came across CAPITAL CRYPTO RECOVER and decided to contact them after reading their reported success record and encouraging client positive reviews and testimonials. Their team guided me through the recovery process, and I was grateful to successfully recover my lost cryptocurrency. The experience was challenging, but I appreciated the assistance and support I received throughout the process. I’m sharing my experience in the hope that it may encourage other victims to carefully research their options and verify any recovery service before proceeding. I will forever be thankful to you CAPITAL CRYPTO RECOVER 📧 [email protected] 🌐 Website: recovercapital.wixsite.com/capital-crypto-rec-1 📧 [email protected] 📞 Call/WhatsApp: +1 (336) 390-6684
09.09.26 23:22 Fraddy Pual

Hearing that these individuals are focusing on other people makes me very sad. I had a similar situation with them and lost a lot of money, but after learning about (Cruxcipherteam @ proton DoT me), whataqq:+168160-15021, telegram: @Cruxcipherteam I was able to get my money back. It's critical that we all be watchful and keep reporting these occurrences.
11.09.26 03:23 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684
11.09.26 03:23 kimberlyhebertt6877

I invested in bitcoin trading After losing $78.4 USDT) linked to a romance fraud scam worth of cryptocurrency through an online investment platform and later discovered it was a scam. After extensive research for recovery options, I contacted CAPITAL CRYPTO RECOVER based on positive client reviews and recommendations. Their professional security team guided me through the recovery process using advanced technology, and I was able to recover my lost cryptocurrency successfully. I am truly grateful for their support and assistance during such a difficult experience. I will advise you to contact CAPITAL CRYPTO RECOVER helped me recover my funds. For anyone facing similar issues, Website: https://recovercapital.wixsite.com/capital-crypto-rec-1 Email: [email protected] Telegram: @Capitalcryptorecover Contact: [email protected] WhatsApp/Text Number: +1 (336) 390-6684
15.09.26 03:35 elioduncan

I fell victim to a freelance web development contract scam that ultimately cost me CAD 4,000. It all started when I came across a job posting on Indeed Canada, a popular online job portal. The position seemed perfect: a freelance web developer role for an established company looking for someone to design and build a fully functional e-commerce website. The job promised a high payment of CAD 20,000 upon successful completion, which sounded like a great opportunity for me to gain experience and earn decent pay.The employer, who introduced himself as a project manager from a "well-known" tech company, was very persuasive and professional in our initial communication. He explained that the project involved developing a user-friendly, responsive online store for their client and that they had a strict timeline to meet. He assured me that the payment would be made promptly after completing the tasks. However, before starting the work, he told me that I would need to pay an upfront fee of CAD 4,000 to cover certain software tools and licensing fees required for the project. This was supposedly a part of their company policy for freelance contractors, ensuring access to their premium resources. The idea of working on a professional project, coupled with the promise of a substantial payout, convinced me to pay the upfront fee.As soon as I made the payment, the project manager became increasingly difficult to reach. He initially responded to my emails and provided some vague instructions on what the project would entail, but as time went on, communication slowed to a complete halt. I never received the required tools or any proper project details. My emails went unanswered, and any attempts to contact the company were met with silence. After waiting for weeks, I realized that I had been scammed.Desperate to recover my money, I turned to TechY Force Cyber Retrieval, a service that specializes in helping victims of online scams. They helped me track the payment and took legal steps to pursue the fraudsters. Through their guidance, I was able to recover the full CAD 4,000. TechY Force Cyber Retrieval worked with my bank to reverse the transaction and liaised with the authorities to trace the scammer's details.This was a hard lesson, and I now know to be highly cautious about online job offers that require upfront payments. It is crucial to research companies thoroughly and avoid any job that seems too good to be true, especially when it involves paying money upfront. WhatsApp https://wa.link/2x6ktp Mail. [email protected]
15.09.26 15:45 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1
15.09.26 15:45 lydiassmith567

HIRE A HACKER YOUR STOLEN CRYPTO RECOVERY / BTC / USDT / ETH WITH THE HELP OF CAPITAL CRYPTO RECOVER. I want to share my experience publicly regarding cryptocurrency wallet recovery, I highly recommend you to contact CAPITAL CRYPTO RECOVER, a professional private investigator in the Bitcoin world. They have a certified expert security team specializing in Bitcoin Recovery Services and have helped many people worldwide recover their lost funds. My wife and I were defrauded by an online manipulator posing as an experienced crypto investment professional. We lost $9.2 Million Stolen BTC in cryptocurrency and were left feeling homeless. After spending hours searching for a reliable crypto recovery service, I discovered CAPITAL CRYPTO RECOVER online. By patiently explaining my situation to their team, I was able to recover all my funds. Remarkably, my money was returned to my wallet in less than 24 hours. I am extremely grateful to CAPITAL CRYPTO RECOVER for their excellent assistance—they truly were a godsend in my difficult situation. If you have fallen victim to a cryptocurrency scam, you can reach CAPITAL CRYPTO RECOVER through the following channels Email: [email protected] OR Call/WhatsApp: +1 (336) 390-6684 Contact: [email protected] Website: https://recovercapital.wixsite.com/capital-crypto-rec-1