Knowledge Distillation: Compressing Trading Models for Low-Latency Deployment

ML asosidagi savdoda aniqlik va kechikish o'rtasidagi ziddiyatga bu blogda allaqachon javob berilgan. Mashinani o'rganish bilan spreadni modellashtirish ikki bosqichli ajratishni tavsiya qiladi: tezkor gradient boosting modeli kechikish muhim bo'lgan real vaqtdagi kotirovkalarni bajaradi, chuqur model esa asinxron ishlaydi va unga ikkilamchi signal beradi yoki parametrlarini moslaydi. Ikki model, ikki soat, bitta tizim.

Bilimlarni distillatsiya qilish shu ziddiyatga boshqa javobdir. Sekin modelni tezkor model bilan birga ishga tushirish o'rniga, undan bir marta, oflayn rejimda tezkor modelni o'rgatish uchun foydalanasiz — student faqat qat'iy label'larni emas, teacher'ning natijalar bo'yicha to'liq ehtimollik taqsimotini o'rganadi, keyin teacher hot path'dan butunlay chiqib ketadi. Inference vaqtida bitta model, asinxron bog'lanish yo'q, eskirish oynasi yo'q.

Qaysi javob ustun kelishini empirik ma'lumot hal qiladi, bu maqola esa hali bunga javob bermaydi. Quyida mexanizm va qarorni belgilaydigan o'lchovlarning aniq ro'yxati beriladi. Bu yerda hech narsa benchmark natijasi emas; odatda raqam bo'lishi kerak bo'lgan joyda nima ishga tushirilishi kerakligini ko'rsatuvchi marker bor.

Avval bitta muhim tuzatish, DeepLOB va order book'dagi deep learning maqolasidan: yuqori classification accuracy avtomatik ravishda foydaga aylanmaydi — bashorat qilingan harakat bid-ask spread'dan oshib ketishi kerak. Shuning uchun distillatsiya tizimini "teacher'ning yo'nalish aniqligini saqlash"ga optimallashtirish noto'g'ri maqsaddir.

Teacher-student freymvorki

Hinton, Vinyals va Dean (2015) taklif qilgan dastlabki formulatsiya sodda. Sizda teacher model $T$ (katta, sekin, aniq) va student model $S$ (kichik, tez, o'qitilishi kerak) bor. Student bir vaqtning o'zida ikkita signaldan o'rganadi:

1. Qat'iy target'lar: ground-truth label'lar $y$ (masalan, narx ko'tarildi yoki tushdi)
2. Yumshoq target'lar: teacher'ning barcha class'lar bo'yicha chiqish ehtimollik taqsimoti $q_T$

Student'ning loss funksiyasi ikkalasini birlashtiradi:

$\mathcal{L} = \alpha \cdot \mathcal{L}_{\text{CE}}(y, \sigma(z_S)) + (1 - \alpha) \cdot T^2 \cdot D_{\text{KL}}\left(\sigma\left(\frac{z_T}{T}\right) \| \sigma\left(\frac{z_S}{T}\right)\right)$

bu yerda $z_T$ va $z_S$ teacher va student logit'lari, $\sigma$ softmax funksiyasi, $T$ temperature parametri, $\alpha$ esa ikki loss komponenti o'rtasidagi muvozanatni boshqaradi.

Nima uchun yumshoq target'lar trading uchun muhim

Uch class'li up/stationary/down mid-price formulasi, $\pm\alpha$ thresholding va hosil bo'ladigan imbalance sabab accuracy emas, weighted F1 haqida hisobot berish kerakligi DeepLOB maqolasida berilgan — shu label sxemasini bu yerda qabul qiling. Distillatsiyaga xos nuqta teacher argmax'dan oldin nima chiqarishidir: qat'iy "up" bitta bit olib yuradi, 0.72/0.21/0.07 esa harakat to'xtab qolishi va deyarli albatta teskari yo'nalmasligini ham bildiradi. Class'lar bo'ylab shu tuzilma qo'shimcha training signal bo'ladi; yumshoq target'larda o'qitilgan student faqat label'larda o'qitilgan ayni student'dan yaxshiroq umumlashishi shuning uchun mumkin.

Bu confidence nimani anglatmasligi haqida ogohlantirish. Softmax output'i calibrated uncertainty emas, 0.55 va 0.85 ni position sizing input'i sifatida ishlatish esa trading uchun conformal prediction rad etadigan qisqa yo'ldir — u sizing'ni interval kengligi, edge ratio va interval nolni kesib o'tganda no-trade filter asosida hosil qiladi, bularning hech birini xom softmax bermaydi. Bu yerda sizing haqidagi da'voni asoslash uchun student calibration'ini teacher calibration'i bilan (reliability diagram, ECE) o'lchab, distillatsiya uni saqlashini ko'rsatish kerak. Bu natija hali maqolada yo'q.

Temperature va yumshoq target'lar

Temperature parametri $T$ ehtimollik taqsimotining "yumshoqligi"ni boshqaradi. $z_i$ logit'lari berilganda, temperature bilan softmax quyidagicha:

$\sigma(z_i; T) = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}$

$T = 1$ bo'lganda (standart softmax), taqsimot cho'qqili bo'ladi — dominant class ehtimollik massasining katta qismini oladi. $T$ oshgani sari taqsimot tekislanib, logit'larning nisbiy kattaliklarini aniqroq ko'rsatadi.

Trading modellari o'rtacha temperature'ni xohlashi mumkin degan asosli fikr bor: moliyaviy bashoratlar image classification'ga qaraganda ancha kam ishonchli, shuning uchun teacher 0.99/0.005/0.005 o'rniga 0.55/0.30/0.15 chiqarishi mumkin; bunda signal yuvilib ketishidan oldin yumshatish uchun cho'qqilik kamroq qoladi. Bu dalil, finding emas — diapazon real ma'lumotdagi sweep'dan, weighted F1 bilan baholangan holda olinishi kerak va rejimga qarab farq qilishi mumkin.

KL divergence hadidagi $T^2$ koeffitsiyenti yuqori temperature'lardagi gradient kattaliklarining kamayishini qoplaydi. U bo'lmasa, $T$ oshgani sari distillatsiya loss'i e'tiborsiz darajada kichik bo'lib qoladi.

Temperature'ni grid search orqali tanlash

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader
from sklearn.metrics import f1_score

def distillation_loss(
    student_logits: torch.Tensor,
    teacher_logits: torch.Tensor,
    labels: torch.Tensor,
    temperature: float,
    alpha: float,
) -> torch.Tensor:
    """Combined hard-target + soft-target distillation loss."""
    hard_loss = F.cross_entropy(student_logits, labels)

    soft_teacher = F.log_softmax(teacher_logits / temperature, dim=-1)
    soft_student = F.log_softmax(student_logits / temperature, dim=-1)

    soft_loss = F.kl_div(
        soft_student,
        soft_teacher,
        log_target=True,
        reduction="batchmean",
    )

    return alpha * hard_loss + (1.0 - alpha) * (temperature ** 2) * soft_loss


def search_temperature(
    teacher: nn.Module,
    student_factory,       # callable returning a fresh student
    train_loader: DataLoader,
    val_loader: DataLoader,
    temperatures: list[float] = [1, 2, 3, 5, 8, 12],
    alpha: float = 0.3,
    epochs: int = 30,
    lr: float = 1e-3,
    device: str = "cuda",
):
    """Grid search over temperature, scored by weighted F1 (not accuracy:
    the up/flat/down label scheme is heavily imbalanced toward flat)."""
    best_f1, best_T, best_student = 0.0, 1.0, None

    for T in temperatures:
        student = student_factory().to(device)
        optimizer = torch.optim.AdamW(student.parameters(), lr=lr)

        for epoch in range(epochs):
            student.train()
            for X, y in train_loader:
                X, y = X.to(device), y.to(device)
                with torch.no_grad():
                    teacher_logits = teacher(X)
                student_logits = student(X)

                loss = distillation_loss(
                    student_logits, teacher_logits, y, T, alpha
                )
                optimizer.zero_grad()
                loss.backward()
                optimizer.step()

        student.eval()
        preds, targets = [], []
        with torch.no_grad():
            for X, y in val_loader:
                preds.append(student(X.to(device)).argmax(dim=-1).cpu())
                targets.append(y)

        f1 = f1_score(
            torch.cat(targets), torch.cat(preds), average="weighted"
        )
        print(f"T={T:>4.1f}  val_weighted_f1={f1:.4f}")
        if f1 > best_f1:
            best_f1, best_T, best_student = f1, T, student

    print(f"\nBest temperature: T={best_T}, val_weighted_f1={best_f1:.4f}")
    return best_T, best_student

Ansambllarni bitta modelga distillatsiya qilish

Quant ensemble inductive bias'larni aralashtiradi: order-book feature'laridagi gradient-boosted tree, so'nggi tick'lar ustidagi 1D-CNN, multi-timeframe window'lar ustidagi transformer va macro factor'lar asosidagi linear model. O'rtacha qiymat har bir a'zoning o'zidan barqarorroq, ammo to'rttasini ishga tushirish latency va cost'ni ko'paytiradi — mashinani o'rganish bilan spreadni modellashtirish maqolasidagi two-stage split bu vaziyatni sekin a'zolarni asinxron side channel'ga tushirish orqali hal qiladi. Distillatsiya esa to'rttasini hot path'dagi bitta student'ga birlashtiradi.

Ensemble teacher'ning output'i uning a'zolari softmax output'larining o'rtacha qiymatidir:

$q_{\text{ensemble}}(x) = \frac{1}{K} \sum_{k=1}^{K} \sigma(z_k(x) / T)$

bu yerda $K$ ensemble a'zolari soni. Student shu o'rtacha taqsimotga qarshi o'qitiladi.

class EnsembleTeacher(nn.Module):
    """Wraps K models, returns averaged logits for distillation."""

    def __init__(self, models: list[nn.Module]):
        super().__init__()
        self.models = nn.ModuleList(models)

    @torch.no_grad()
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        logits = torch.stack([m(x) for m in self.models], dim=0)
        return logits.mean(dim=0)   # average logits, not softmax


class TradingStudent(nn.Module):
    """Lightweight MLP for sub-millisecond inference."""

    def __init__(self, input_dim: int, hidden: int = 64, n_classes: int = 3):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden),
            nn.ReLU(),
            nn.BatchNorm1d(hidden),
            nn.Linear(hidden, hidden),
            nn.ReLU(),
            nn.BatchNorm1d(hidden),
            nn.Linear(hidden, n_classes),
        )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.net(x)

Parametrlar sonidagi asimmetriya butun nuqtadir: 64 ta hidden unit'li ikki qatlamli MLP 60 feature'li, 3 class'li vazifa uchun taxminan 8 000 parametrga ega, ensemble'ning jami esa millionlarga boradi.

Student nimani saqlaydi va nimani yo'qotadi

Bu eng muhim empirik savol, maqola esa unga javob bermaydi. Intuitsiya shuki, student in-distribution holatda ensemble'ni kuzatadi, ammo ensemble xilma-xilligi ish beradigan stressli rejimlarda ortda qoladi — lekin retention ko'rsatkichi faqat real order-book ma'lumotlarida, rejimlar bo'yicha ajratib, weighted F1 sifatida o'lchangandagina mazmunli. Tinch kunlarda barqaror qolib, liquidation cascade paytida qulaydigan student sekin yomonlashadigan student'dan boshqa mahsulotdir, aggregate raqam esa ularni ajrata olmaydi.

Oldindan da'vo qilish o'rniga, shu o'lchov asosida uchta yumshatish usulini sinash kerak:

1. Student bo'shliq ochilishi kutiladigan rejimlarni ko'rishi uchun distillatsiya to'plamiga stressli davrlarni qo'shing.
2. Feature-based distillatsiya — faqat final output'larni emas, oraliq representation'larni ham moslang.
3. Student'ga yordamchi regime head qo'shib, rejimni hisobga oladigan feature'larni umumiy trunk'ga majburan kiriting.

Self-distillatsiya: student teacher'ga aylanganda

Self-distillatsiya — model bilimni o'zidan distillatsiya qiladigan usul.

Qayta tug'ilgan tarmoqlar (BAN)

Teacher bilan bir xil architecture'ga ega student'ni o'qing. "Qayta tug'ilgan" student ko'pincha original'dan ustun keladi va jarayon takrorlanadi:

$M_0 \xrightarrow{\text{distill}} M_1 \xrightarrow{\text{distill}} M_2 \xrightarrow{\text{distill}} \cdots$

Har bir avlod oldingi avlodning soft target'larida o'qitiladi, odatda bir necha avloddan keyin yutuq to'yinganlikka yetadi. Trading modellari uchun architecture nuqtayi nazaridan bu hech narsa talab qilmaydi — yangi feature'lar yo'q, yangi data yo'q, faqat boshqa training procedure — demak sinash arzon va uni sinamasdan hisobot berishga bahona yo'q.

Chuqurlik bo'yicha self-distillatsiya

Oraliq layer'larda auxiliary classifier'lar ulang. Eng chuqur exit sayozroq exit'lar uchun teacher bo'lib xizmat qiladi. Inference paytida exit tanlaysiz: pastroq latency uchun sayoz, maksimal accuracy uchun chuqur.

Bu trading tizimiga eng mos g'oya, chunki exit depth runtime latency knob'iga aylanadi: training vaqtida bitta architecture'ga bog'lanish o'rniga, bitta o'qitilgan network turli budget'larni qamrab oladi. Book tez harakat qilganda sayoz exit'ni tanlab, yomonroq posterior'ni qabul qilasiz; sokin paytda to'liq depth uchun haq to'laysiz. Accuracy-per-exit va latency-per-exit curve'larining ikkalasini ham o'lchash mumkin, ularning kesishishi knob kerakmi-yo'qmi hal qiladi.

class SelfDistillingNet(nn.Module):
    """Network with early-exit classifiers for variable-latency inference."""

    def __init__(self, input_dim: int, n_classes: int = 3):
        super().__init__()
        self.block1 = nn.Sequential(
            nn.Linear(input_dim, 128), nn.ReLU(), nn.BatchNorm1d(128)
        )
        self.block2 = nn.Sequential(
            nn.Linear(128, 64), nn.ReLU(), nn.BatchNorm1d(64)
        )
        self.block3 = nn.Sequential(
            nn.Linear(64, 32), nn.ReLU(), nn.BatchNorm1d(32)
        )

        self.exit1 = nn.Linear(128, n_classes)
        self.exit2 = nn.Linear(64, n_classes)
        self.exit3 = nn.Linear(32, n_classes)  # final exit

    def forward(
        self, x: torch.Tensor, exit_layer: int = 3
    ) -> torch.Tensor:
        h1 = self.block1(x)
        if exit_layer == 1:
            return self.exit1(h1)

        h2 = self.block2(h1)
        if exit_layer == 2:
            return self.exit2(h2)

        h3 = self.block3(h2)
        return self.exit3(h3)

    def forward_all_exits(self, x: torch.Tensor):
        """Return logits from all exits (for self-distillation training)."""
        h1 = self.block1(x)
        h2 = self.block2(h1)
        h3 = self.block3(h2)
        return self.exit1(h1), self.exit2(h2), self.exit3(h3)


def self_distillation_step(
    model: SelfDistillingNet,
    x: torch.Tensor,
    y: torch.Tensor,
    temperature: float = 4.0,
    alpha: float = 0.5,
) -> torch.Tensor:
    """One training step with self-distillation from deepest exit."""
    logits_1, logits_2, logits_3 = model.forward_all_exits(x)

    loss_hard = F.cross_entropy(logits_3, y)

    loss_distill_1 = distillation_loss(
        logits_1, logits_3.detach(), y, temperature, alpha
    )
    loss_distill_2 = distillation_loss(
        logits_2, logits_3.detach(), y, temperature, alpha
    )

    return loss_hard + 0.5 * loss_distill_1 + 0.5 * loss_distill_2

Inference budgeti qayerdan keladi

Distillatsiya faqat inference qat'iy budget ichiga sig'ishi kerak bo'lgandagina muhim, to'liq tick-to-trade ladder esa — NIC-to-userspace, kernel bypass, 100 µs dan kichik jami vaqt va FPGA hamda shared memory'ni majbur qiladigan 10 µs dan kichik tier — algorithmic trading'dagi data va communication maqolasida allaqachon berilgan. Ladder ochiq qoldirgan qator model inference'idir va distillatsiya aynan shu qatorni to'ldirishga urinadi.

Qolgan qatorlarni model-class latency table bilan to'ldirishga shoshilmang. Mashinani o'rganish bilan spreadni modellashtirish GBM va deep-learning taqqoslashini hamda raqamlardan muhimroq ogohlantirishni allaqachon e'lon qilgan: latency implementation'ga bog'liq, ayni LightGBM modeli Python'dan har bir row uchun o'nlab microsecond olsa, compiled predictor'dan bir necha microsecond oladi. Bu yerdagi har qanday latency da'vosi framework, core va batch size'ni ko'rsatishi kerak, aks holda u shovqin.

GPU'lar uchun alohida aytganda: qurilma umuman foyda berishidan oldin har bir launch'ning fixed overhead'i amortizatsiya qilinishi kerak, single-row inference esa roofline ridge'ining juda chapida joylashadi va u yerda bu sodir bo'lmaydi. GPU qachon o'zini oqlashi shu amortizatsiya curve'ini batch sweep bilan, discrete PCIe card ridge'ni yanada o'ngga surishini ham hisobga olgan holda to'g'ri o'lchaydi — xotiradan aytilgan constant'ga ishonish o'rniga shuni o'qing.

Distillatsiyadan keyingi quantization

Distillatsiya qilingan student'ni yana siqish mumkin: INT8 weight'lar (AVX-512 VNNI bilan CPU'da taxminan 2x), multiply'larni add'ga aylantiradigan binary/ternary weight'lar va nolga yaqin hisoblashni o'tkazib yuboradigan pruning.

Jozibali da'vo shuki, distillatsiyadan keyingi quantization faqat quantization'ning o'zidan ko'ra ko'proq accuracy'ni saqlaydi, chunki student allaqachon ixcham representation'ni o'rgangan. Buni o'lchamasdan production'ga chiqarmang. GPU precision tuzog'i kamaytirilgan numeric precision bo'yicha blogning qat'iy pozitsiyasidir: u jimgina ishonarli ko'ringan axlat qaytargan, fast path'ni chiqarishga yaroqli qilgan narsa esa da'vo emas, miqdoriy equivalence gate bo'lgan — siljigan fill'lar, bps'dagi PnL delta. Bu gate FP32 student'ga nisbatan o'lchanmaguncha INT8 student boshqa modeldir.

import torch.quantization as quant

def quantize_student(student: nn.Module, calibration_loader: DataLoader):
    """Post-training static quantization for CPU deployment."""
    student.cpu()
    student.eval()
    student.qconfig = quant.get_default_qconfig("x86")

    student_prepared = quant.prepare(student)

    with torch.no_grad():
        for X, _ in calibration_loader:
            student_prepared(X)

    student_quantized = quant.convert(student_prepared)
    return student_quantized

FPGA deployment: distillatsiyadan bitstream'га pipeline

FPGA'lar latency ladder ichidagi 10 µs dan kichik tier'dir, Tbricks/Broadridge review esa ularni production'da kernel-bypass NIC'lar bilan birga qamrab oladi — deterministic latency, OS jitter yo'q, network stack bilan bir joyda. Bu blogda hech qayerda distillatsiya qilingan model ularning biriga qanday joylashtirilishi yoritilmagan.

DeepLOB production notes ONNX/TensorRT, INT8 quantization va FPGA deployment'ni uchta variant sifatida sanab, shu yerda to'xtaydi. Uchinchisi quyidagicha kengayadi:

1. Train ensemble teacher (offline, GPU cluster, hours/days)
       |
2. Distill to small MLP student (offline, single GPU, minutes)
       |
3. Quantize student to INT8 / fixed-point (offline, CPU)
       |
4. Convert to HLS (High-Level Synthesis) or RTL
       |
5. Synthesize FPGA bitstream (offline, hours)
       |
6. Deploy to FPGA card in production server
       |
7. Inference: market data -> FPGA -> trading signal

Asosiy cheklov shuki, model qurilmaning logic element'lariga — LUT, DSP slice va block RAM'ga — sig'ishi kerak. O'lchov emas, order-of-magnitude budget sifatida: 64 hidden unit va INT8 weight'li 2-layer MLP har bir inference uchun taxminan 8 000 multiply-accumulate va ~16 KB weight talab qiladi, bu o'rta darajadagi part'ning kichik qismi. Distillatsiya aynan shu yerda foydasini ko'rsatadi — ensemble teacher hech qanday budget'ga sig'maydi, student esa limitga yaqin ham emas.

PyTorch/ONNX'ni synthesis qilinadigan hardware'ga avtomatlashtirib o'tkazadigan vositalar qatoriga AMD/Xilinx Vitis AI, hls4ml (CERN'dan) va FINN (Xilinx Research'dan) kiradi.

Misol: hls4ml conversion

import hls4ml
import onnx

dummy_input = torch.randn(1, 60)  # 60 input features
torch.onnx.export(student, dummy_input, "student.onnx", opset_version=13)

hls_config = hls4ml.utils.config_from_onnx_model(
    onnx.load("student.onnx"),
    granularity="name",
    default_precision="ap_fixed<16,8>",
    default_reuse_factor=1,          # full parallelism
)

hls_model = hls4ml.converters.convert_from_onnx_model(
    "student.onnx",
    hls_config=hls_config,
    output_dir="hls_student",
    backend="VivadoAccelerator",
    board="alveo-u250",
)

hls_model.compile()
hls_model.build(csim=True, synth=True)

hls_model.report()

hls_model.report() berilgan model, board, precision va reuse factor uchun resource va latency raqamlarining yagona ishonchli manbaidir — raqamlar faqat default_reuse_factor o'zgarganda ham sezilarli siljiydi. Uni ishga tushirmasdan "typical" synthesis table keltirish taxmin qilishdir.

Amaliy mulohazalar

Teacher logit'larini oldindan hisoblash

Distillatsiya butun training set bo'yicha teacher prediction'larini talab qiladi — ataylab to'lashga arziydigan bir martalik offline cost: ensemble'ni bir marta ishga tushiring, logit'larni saqlang, student'larni cache'ga qarshi o'qiting. Shundan keyin temperature sweep'lar va architecture search'lar teacher forward pass'larida qo'shimcha cost talab qilmaydi; yuqoridagi sweep'larni amaliy qiladigan narsa ham shu.

Distillatsiyaga xos yagona monitor

Feature pipeline gigiyenasi, z-score parameter'lari siljishi sababli rolling normalization, input distribution shift monitoring'i va rejim trigger qiladigan retraining DeepLOB production section da yoritilgan va bu yerda o'zgarishsiz qo'llanadi.

Distillatsiyaga xos monitor — live data'dagi teacher-student KL divergence. Teacher offline mavjud bo'lib turadi; uni live input'larning bir sample'ida ishga tushirib, taqsimotlarni solishtiring. KL oshishi student approximation'i distillatsiya qilinmagan rejimlarda yomonlashayotganini anglatadi — u accuracy'dan oldin signal beradi, chunki label'larni kutmaydi. Retraining threshold'i ma'lum yaxshi va ma'lum yomon davrlarda kuzatilgan KL asosida calibrated qilinishi kerak; a priori tanlansa, u o'zboshimcha bo'ladi.

Qachon distillatsiya qilmaslik kerak

• Teacher allaqachon kichik (linear model, shallow GBM): distillatsiya hech qanday compression bermay, pipeline'ga qo'shimcha bosqich qo'shadi.
• Latency cheklov emas (daily rebalancing, kun oxiri signal'lari): teacher'ni deploy qiling.
• Interpretability tezlikdan ustun: distillatsiya qilingan network o'zi almashtirgan tree ensemble'dan ko'ra qiyinroq tushuntiriladi.
• Two-stage split allaqachon ishlaydi: agar spread-modeling architecture dagi asinxron sekin model natija berayotgan bo'lsa, ishlayotgan tizimni almashtirishni oqlashdan oldin distillatsiya uni o'lchangan taqqoslashda yengishi kerak.

Xulosa

Distillatsiya two-stage fast/slow split'ga izchil alternativadir: offline ko'tara oladigan eng yaxshi teacher'ni o'qing, uning soft-target structure'ini hot path uchun yetarlicha kichik student'ga o'tkazing, quantize qiling va CPU yoki FPGA'ga deploy qiling. Depth-wise variant bundan ham uzoqqa borib, latency'ni training-time tanlovi emas, runtime tanloviga aylantiradi.

Bu maqola ataylab shuni da'vo qilmaydi: bularning birortasi blogda allaqachon e'lon qilingan yechimdan ustun. Bunday xulosa uchun real order-book data'da uchta o'lchov kerak: rejimlar bo'yicha ajratilgan student va ensemble weighted F1 retention curve'i, temperature sweep'i va GPU precision tuzog'i uslubidagi INT8 parity gate'i. Ular mavjud bo'lmaguncha, bu texnika tavsifi, deploy qilish tavsiyasi emas.

Manbalar

1. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531

2. Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018). Born-Again Neural Networks. ICML. arXiv:1805.04770

3. Zhang, L., Song, J., Gao, A., Chen, J., Bao, C., & Ma, K. (2019). Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation. ICCV. arXiv:1905.08094

4. Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2015). FitNets: Hints for Thin Deep Nets. ICLR. arXiv:1412.6550

5. Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021). Knowledge Distillation: A Survey. International Journal of Computer Vision, 129, 1789-1819. arXiv:2006.05525

6. Duarte, J., et al. (2018). Fast Inference of Deep Neural Networks in FPGAs for Particle Physics (hls4ml). Journal of Instrumentation, 13, P07027. arXiv:1804.06913

7. Umuroglu, Y., et al. (2017). FINN: A Framework for Fast, Scalable Binarized Neural Network Inference. FPGA '17. arXiv:1612.07119

8. Zhang, Z., Zohren, S., & Roberts, S. (2019). DeepLOB: Deep Convolutional Neural Networks for Limit Order Books. IEEE Transactions on Signal Processing, 67(11), 3001-3012. arXiv:1808.03668