Experiments

This section documents various experiments conducted in the project, their results, and conclusions.

Analysis: All comparisons are by relative time (minutes from run start). Use scripts/analyze_experiment_by_relative_time.py with two or more runs (e.g. uni_5 uni_7 or uni_12 uni_13 uni_14). The script prints per-race tables (best/mean/std, finish rate, first finish) from Race/eval_race_time_* and Race/explo_race_time_*, then scalar metrics (alltime_min_ms_*, loss, Q, GPU %). For runs logged before the learner_process fix, prefer per-race tables for race-time comparison.

Comparison plots: Each experiment page embeds JPG graphs (one metric per graph, runs as lines) next to the metric they illustrate in “Detailed TensorBoard Metrics Analysis”. Each image has an alt text (caption) describing the metric and runs. The image files (exp_*.jpg in docs/source/_static/) are generated by running python scripts/generate_experiment_plots.py (with TensorBoard logs present, e.g. tensorboard/uni_12) and should be committed so the built docs include the plots. Use the project venv; if activation fails, run .venv\Scripts\python.exe scripts/generate_experiment_plots.py (Windows).

Contents

• Batch Size and Running Speed
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Engineered rewards (speedslide, neoslide)
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Extended Training, One vs Two Maps
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Temporal Mini-Race Duration
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Epsilon-Greedy Exploration
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Network Size and Long Training
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Replay pretrain roadmap
  • Track representation for RL (context for pretrain)
  • Key insights (from knowledge base)
  • Simplest experiments to run first (minimal setup)
  • Roadmap: experiments from simplest to most complex
  • Summary table: pretrain → IQN pipeline
  • Practical order of experiments (revised)
  • References (from knowledge base)
• Visual backbone pretraining
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• BC pretraining
  • Experiment Overview
  • Results
  • Run Analysis
  • Detailed TensorBoard Metrics Analysis
  • Configuration Changes
  • Hardware
  • Conclusions
  • Recommendations
• Pretrain
  • Contents
• IQN architecture
  • What we use
  • Training loop (data flow)
  • Why distributional (quantiles)
  • Why dueling (V + A)
  • Why Double DQN (optional)
  • Why mini-races (clipped horizon)
  • Normalization
  • Pretrained encoder (Level 0 / BC)
  • Overview: inputs and outputs
  • High-level diagram (main blocks)
  • Block details
  • Other implementation details
  • Config parameters
  • See also
• IQN model experiments
  • Experiment 1: Double DQN (use_ddqn)
  • Experiment 2: iqn_embedding_dimension (128 vs 64)
  • Experiment 3: Image dimensions (W_downsized / H_downsized 256 vs 128)
  • Experiment 4: Image dimensions 64×64 vs 128×128 (downsized model)
  • Experiment 5: Image dimensions 64×64 vs 128×128 (embedding 128 — isolates resolution)
  • Analysis tools (all IQN experiments)