{"schema_version":1,"project":"GenomicTouchstone","protocol":"FT","as_of":"2026-10-18","models_count":43,"settings_count":100,"settings_by_modality":{"DNA":52,"Protein":15,"RNA":33},"method":"Rank full-precision accepted means within each setting, higher is better; exact ties receive average ranks. Average these ranks with equal setting weights, then rank ascending. Overall includes 52 DNA, 33 RNA and 15 protein settings.","scope_note":"Current 100-setting FT main table. Includes the restored OpenVaccine and Tc-riboswitches random settings and PhyloGPN. This website snapshot is separate from the earlier frozen 98-setting manuscript aggregate. LP, three independent long-range settings and task-specific references are excluded.","caveat":"Accepted historical learning-rate, seed and precision choices are retained; some were informed by test results. Seed SD describes the accepted repetitions, not a confidence interval or an unbiased fixed-seed stability estimate. Settings include related datasets and split protocols.","snapshot":{"results_manifest_sha256":"948b17d6b3bf01c466b10621ade1e2cd4fd2bd88de7773952b0d56c03c0b7e7a","details_sha256":"12ebffebcfd8bddd06a166007e7c30f60ed0863a9c12a6cfd437e8a8ba017aa8","accepted_release":"20261018_100814_phylogpn_random"},"models":[{"id":"alphagenome","name":"AlphaGenome","baseline":false},{"id":"baseline_cnn","name":"CNN","baseline":true},{"id":"baseline_lstm","name":"LSTM","baseline":true},{"id":"borzoi","name":"Borzoi","baseline":false},{"id":"caduceus_ph_seqlen_131k_d_model_256","name":"Caduceus-Ph","baseline":false},{"id":"caduceus_ps_seqlen_131k_d_model_256","name":"Caduceus-Ps","baseline":false},{"id":"cnn-56m-random-init","name":"CNN-56M-Random-Init","baseline":true},{"id":"dnabert-3","name":"DNABERT-3mer","baseline":false},{"id":"dnabert-4","name":"DNABERT-4mer","baseline":false},{"id":"dnabert-5","name":"DNABERT-5mer","baseline":false},{"id":"dnabert-6","name":"DNABERT-6mer","baseline":false},{"id":"dnabert2","name":"DNABERT2","baseline":false},{"id":"enformer","name":"Enformer","baseline":false},{"id":"evo2_1b_base","name":"Evo2-1B-Base","baseline":false},{"id":"gena-lm-bert-base","name":"Gena-LM-Base","baseline":false},{"id":"gena-lm-bert-base-lastln-t2t","name":"Gena-LM-Base-Lastln-T2T","baseline":false},{"id":"gena-lm-bert-base-t2t","name":"Gena-LM-Base-T2T","baseline":false},{"id":"gena-lm-bert-base-t2t-multi","name":"Gena-LM-Base-T2T-Multi","baseline":false},{"id":"gena-lm-bert-large-t2t","name":"Gena-LM-Large","baseline":false},{"id":"gena-lm-bigbird-base-t2t","name":"Gena-LM-BigBird","baseline":false},{"id":"generator_eukaryote_1_2b","name":"Generator-1.2B","baseline":false},{"id":"phylogpn","name":"PhyloGPN","baseline":false},{"id":"grover","name":"Grover","baseline":false},{"id":"hyena_large_1m","name":"Hyena-Large","baseline":false},{"id":"hyena_medium_160k","name":"Hyena-Medium","baseline":false},{"id":"ntv2-50m-random-init","name":"NTv2-50M-Random-Init","baseline":true},{"id":"ntv3-100m-pre","name":"NTv3-100M","baseline":false},{"id":"ntv3-650m-pre","name":"NTv3-650M","baseline":false},{"id":"ntv3-8m-pre","name":"NTv3-8M","baseline":false},{"id":"nucleotide-transformer-2.5b-1000g","name":"NT-2.5B-1000g","baseline":false},{"id":"nucleotide-transformer-2.5b-multi-species","name":"NT-2.5B-Multi","baseline":false},{"id":"nucleotide-transformer-500m-1000g","name":"NT-500M-1000g","baseline":false},{"id":"nucleotide-transformer-500m-human-ref","name":"NT-500M-Human-Ref","baseline":false},{"id":"nucleotide-transformer-v2-100m-multi-species","name":"NTv2-100M-Multi","baseline":false},{"id":"nucleotide-transformer-v2-250m-multi-species","name":"NTv2-250M-Multi","baseline":false},{"id":"nucleotide-transformer-v2-500m-multi-species","name":"NTv2-500M-Multi","baseline":false},{"id":"nucleotide-transformer-v2-50m-multi-species","name":"NTv2-50M-Multi","baseline":false},{"id":"orthrus-4track","name":"Orthrus-4Track","baseline":false},{"id":"rinalmo","name":"RiNALMo","baseline":false},{"id":"rnabert","name":"RNA-BERT","baseline":false},{"id":"rnaernie","name":"RNAErnie","baseline":false},{"id":"rnafm","name":"RNAFM","baseline":false},{"id":"splicebert","name":"SpliceBERT","baseline":false}],"summary":{"Overall":[{"model":"nucleotide-transformer-2.5b-multi-species","mean_rank":8.92,"settings":100,"top_scores":5,"rank":1.0},{"model":"ntv3-650m-pre","mean_rank":9.56,"settings":100,"top_scores":9,"rank":2.0},{"model":"generator_eukaryote_1_2b","mean_rank":9.86,"settings":100,"top_scores":19,"rank":3.0},{"model":"alphagenome","mean_rank":10.02,"settings":100,"top_scores":30,"rank":4.0},{"model":"nucleotide-transformer-v2-500m-multi-species","mean_rank":10.81,"settings":100,"top_scores":4,"rank":5.0},{"model":"ntv3-100m-pre","mean_rank":13.78,"settings":100,"top_scores":0,"rank":6.0},{"model":"nucleotide-transformer-v2-250m-multi-species","mean_rank":14.08,"settings":100,"top_scores":0,"rank":7.0},{"model":"nucleotide-transformer-2.5b-1000g","mean_rank":14.2,"settings":100,"top_scores":2,"rank":8.0},{"model":"rinalmo","mean_rank":15.0,"settings":100,"top_scores":3,"rank":9.0},{"model":"ntv3-8m-pre","mean_rank":15.22,"settings":100,"top_scores":2,"rank":10.0},{"model":"borzoi","mean_rank":15.31,"settings":100,"top_scores":4,"rank":11.0},{"model":"splicebert","mean_rank":16.71,"settings":100,"top_scores":5,"rank":12.0},{"model":"nucleotide-transformer-v2-100m-multi-species","mean_rank":16.87,"settings":100,"top_scores":0,"rank":13.0},{"model":"gena-lm-bert-large-t2t","mean_rank":16.92,"settings":100,"top_scores":0,"rank":14.0},{"model":"nucleotide-transformer-v2-50m-multi-species","mean_rank":19.27,"settings":100,"top_scores":0,"rank":15.0},{"model":"gena-lm-bert-base-lastln-t2t","mean_rank":19.48,"settings":100,"top_scores":0,"rank":16.0},{"model":"enformer","mean_rank":20.25,"settings":100,"top_scores":3,"rank":17.0},{"model":"nucleotide-transformer-500m-human-ref","mean_rank":21.5,"settings":100,"top_scores":1,"rank":18.0},{"model":"rnaernie","mean_rank":21.51,"settings":100,"top_scores":1,"rank":19.0},{"model":"evo2_1b_base","mean_rank":22.17,"settings":100,"top_scores":1,"rank":20.0},{"model":"rnafm","mean_rank":22.36,"settings":100,"top_scores":2,"rank":21.0},{"model":"grover","mean_rank":22.47,"settings":100,"top_scores":1,"rank":22.5},{"model":"nucleotide-transformer-500m-1000g","mean_rank":22.47,"settings":100,"top_scores":1,"rank":22.5},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean_rank":23.03,"settings":100,"top_scores":0,"rank":24.0},{"model":"gena-lm-bigbird-base-t2t","mean_rank":24.23,"settings":100,"top_scores":0,"rank":25.0},{"model":"phylogpn","mean_rank":24.69,"settings":100,"top_scores":0,"rank":26.0},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean_rank":25.1,"settings":100,"top_scores":0,"rank":27.0},{"model":"dnabert-5","mean_rank":25.33,"settings":100,"top_scores":1,"rank":28.0},{"model":"dnabert-4","mean_rank":25.49,"settings":100,"top_scores":1,"rank":29.0},{"model":"dnabert-3","mean_rank":25.83,"settings":100,"top_scores":0,"rank":30.0},{"model":"dnabert-6","mean_rank":26.14,"settings":100,"top_scores":1,"rank":31.0},{"model":"hyena_medium_160k","mean_rank":26.71,"settings":100,"top_scores":0,"rank":32.0},{"model":"gena-lm-bert-base-t2t-multi","mean_rank":27.44,"settings":100,"top_scores":1,"rank":33.0},{"model":"hyena_large_1m","mean_rank":27.99,"settings":100,"top_scores":2,"rank":34.0},{"model":"gena-lm-bert-base-t2t","mean_rank":28.02,"settings":100,"top_scores":0,"rank":35.0},{"model":"dnabert2","mean_rank":28.43,"settings":100,"top_scores":0,"rank":36.0},{"model":"gena-lm-bert-base","mean_rank":28.73,"settings":100,"top_scores":0,"rank":37.0},{"model":"baseline_lstm","mean_rank":29.59,"settings":100,"top_scores":1,"rank":38.0},{"model":"orthrus-4track","mean_rank":32.14,"settings":100,"top_scores":0,"rank":39.0},{"model":"baseline_cnn","mean_rank":33.44,"settings":100,"top_scores":0,"rank":40.0},{"model":"cnn-56m-random-init","mean_rank":33.505,"settings":100,"top_scores":0,"rank":41.0},{"model":"ntv2-50m-random-init","mean_rank":33.52,"settings":100,"top_scores":0,"rank":42.0},{"model":"rnabert","mean_rank":37.905,"settings":100,"top_scores":0,"rank":43.0}],"DNA":[{"model":"alphagenome","mean_rank":6.1923076923076925,"settings":52,"top_scores":28,"rank":1.0},{"model":"nucleotide-transformer-2.5b-multi-species","mean_rank":8.26923076923077,"settings":52,"top_scores":3,"rank":2.0},{"model":"nucleotide-transformer-v2-500m-multi-species","mean_rank":9.788461538461538,"settings":52,"top_scores":3,"rank":3.0},{"model":"generator_eukaryote_1_2b","mean_rank":10.01923076923077,"settings":52,"top_scores":6,"rank":4.0},{"model":"borzoi","mean_rank":10.73076923076923,"settings":52,"top_scores":2,"rank":5.5},{"model":"ntv3-650m-pre","mean_rank":10.73076923076923,"settings":52,"top_scores":0,"rank":5.5},{"model":"enformer","mean_rank":13.076923076923077,"settings":52,"top_scores":2,"rank":7.0},{"model":"nucleotide-transformer-v2-250m-multi-species","mean_rank":13.903846153846153,"settings":52,"top_scores":0,"rank":8.0},{"model":"nucleotide-transformer-2.5b-1000g","mean_rank":14.288461538461538,"settings":52,"top_scores":0,"rank":9.0},{"model":"gena-lm-bert-large-t2t","mean_rank":14.807692307692308,"settings":52,"top_scores":0,"rank":10.0},{"model":"splicebert","mean_rank":15.25,"settings":52,"top_scores":1,"rank":11.0},{"model":"ntv3-100m-pre","mean_rank":16.51923076923077,"settings":52,"top_scores":0,"rank":12.0},{"model":"ntv3-8m-pre","mean_rank":18.115384615384617,"settings":52,"top_scores":1,"rank":13.0},{"model":"gena-lm-bert-base-lastln-t2t","mean_rank":18.23076923076923,"settings":52,"top_scores":0,"rank":14.0},{"model":"nucleotide-transformer-v2-100m-multi-species","mean_rank":18.48076923076923,"settings":52,"top_scores":0,"rank":15.0},{"model":"rinalmo","mean_rank":19.096153846153847,"settings":52,"top_scores":0,"rank":16.0},{"model":"grover","mean_rank":21.423076923076923,"settings":52,"top_scores":0,"rank":17.0},{"model":"nucleotide-transformer-v2-50m-multi-species","mean_rank":21.76923076923077,"settings":52,"top_scores":0,"rank":18.0},{"model":"rnafm","mean_rank":21.884615384615383,"settings":52,"top_scores":1,"rank":19.0},{"model":"rnaernie","mean_rank":22.326923076923077,"settings":52,"top_scores":0,"rank":20.0},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean_rank":22.71153846153846,"settings":52,"top_scores":0,"rank":21.5},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean_rank":22.71153846153846,"settings":52,"top_scores":0,"rank":21.5},{"model":"gena-lm-bigbird-base-t2t","mean_rank":23.384615384615383,"settings":52,"top_scores":0,"rank":23.0},{"model":"phylogpn","mean_rank":23.76923076923077,"settings":52,"top_scores":0,"rank":24.0},{"model":"nucleotide-transformer-500m-human-ref","mean_rank":24.346153846153847,"settings":52,"top_scores":0,"rank":25.0},{"model":"dnabert-4","mean_rank":24.557692307692307,"settings":52,"top_scores":1,"rank":26.0},{"model":"evo2_1b_base","mean_rank":24.576923076923077,"settings":52,"top_scores":0,"rank":27.0},{"model":"nucleotide-transformer-500m-1000g","mean_rank":24.846153846153847,"settings":52,"top_scores":1,"rank":28.0},{"model":"dnabert-5","mean_rank":24.96153846153846,"settings":52,"top_scores":0,"rank":29.0},{"model":"dnabert-6","mean_rank":25.53846153846154,"settings":52,"top_scores":0,"rank":30.0},{"model":"gena-lm-bert-base-t2t-multi","mean_rank":25.71153846153846,"settings":52,"top_scores":1,"rank":31.0},{"model":"dnabert-3","mean_rank":26.326923076923077,"settings":52,"top_scores":0,"rank":32.0},{"model":"gena-lm-bert-base-t2t","mean_rank":27.307692307692307,"settings":52,"top_scores":0,"rank":33.0},{"model":"dnabert2","mean_rank":27.51923076923077,"settings":52,"top_scores":0,"rank":34.0},{"model":"hyena_medium_160k","mean_rank":27.884615384615383,"settings":52,"top_scores":0,"rank":35.0},{"model":"gena-lm-bert-base","mean_rank":28.076923076923077,"settings":52,"top_scores":0,"rank":36.0},{"model":"hyena_large_1m","mean_rank":29.03846153846154,"settings":52,"top_scores":2,"rank":37.0},{"model":"baseline_lstm","mean_rank":31.442307692307693,"settings":52,"top_scores":0,"rank":38.0},{"model":"baseline_cnn","mean_rank":32.90384615384615,"settings":52,"top_scores":0,"rank":39.0},{"model":"orthrus-4track","mean_rank":33.0,"settings":52,"top_scores":0,"rank":40.0},{"model":"cnn-56m-random-init","mean_rank":34.81730769230769,"settings":52,"top_scores":0,"rank":41.0},{"model":"ntv2-50m-random-init","mean_rank":36.80769230769231,"settings":52,"top_scores":0,"rank":42.0},{"model":"rnabert","mean_rank":38.85576923076923,"settings":52,"top_scores":0,"rank":43.0}],"RNA":[{"model":"ntv3-650m-pre","mean_rank":8.515151515151516,"settings":33,"top_scores":5,"rank":1.0},{"model":"rinalmo","mean_rank":8.696969696969697,"settings":33,"top_scores":3,"rank":2.0},{"model":"nucleotide-transformer-2.5b-multi-species","mean_rank":10.909090909090908,"settings":33,"top_scores":0,"rank":3.0},{"model":"ntv3-100m-pre","mean_rank":11.696969696969697,"settings":33,"top_scores":0,"rank":4.0},{"model":"generator_eukaryote_1_2b","mean_rank":11.757575757575758,"settings":33,"top_scores":10,"rank":5.0},{"model":"ntv3-8m-pre","mean_rank":12.606060606060606,"settings":33,"top_scores":1,"rank":6.0},{"model":"nucleotide-transformer-v2-500m-multi-species","mean_rank":13.454545454545455,"settings":33,"top_scores":0,"rank":7.0},{"model":"alphagenome","mean_rank":13.484848484848484,"settings":33,"top_scores":2,"rank":8.0},{"model":"nucleotide-transformer-2.5b-1000g","mean_rank":15.575757575757576,"settings":33,"top_scores":0,"rank":9.5},{"model":"nucleotide-transformer-v2-250m-multi-species","mean_rank":15.575757575757576,"settings":33,"top_scores":0,"rank":9.5},{"model":"splicebert","mean_rank":16.181818181818183,"settings":33,"top_scores":3,"rank":11.0},{"model":"nucleotide-transformer-v2-100m-multi-species","mean_rank":16.424242424242426,"settings":33,"top_scores":0,"rank":12.0},{"model":"nucleotide-transformer-v2-50m-multi-species","mean_rank":17.636363636363637,"settings":33,"top_scores":0,"rank":13.0},{"model":"evo2_1b_base","mean_rank":18.09090909090909,"settings":33,"top_scores":0,"rank":14.0},{"model":"nucleotide-transformer-500m-human-ref","mean_rank":19.606060606060606,"settings":33,"top_scores":1,"rank":15.0},{"model":"rnaernie","mean_rank":19.727272727272727,"settings":33,"top_scores":1,"rank":16.5},{"model":"rnafm","mean_rank":19.727272727272727,"settings":33,"top_scores":1,"rank":16.5},{"model":"gena-lm-bert-large-t2t","mean_rank":20.12121212121212,"settings":33,"top_scores":0,"rank":18.0},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean_rank":20.545454545454547,"settings":33,"top_scores":0,"rank":19.0},{"model":"nucleotide-transformer-500m-1000g","mean_rank":21.03030303030303,"settings":33,"top_scores":0,"rank":20.0},{"model":"gena-lm-bert-base-lastln-t2t","mean_rank":21.333333333333332,"settings":33,"top_scores":0,"rank":21.0},{"model":"borzoi","mean_rank":22.606060606060606,"settings":33,"top_scores":1,"rank":22.0},{"model":"hyena_medium_160k","mean_rank":23.181818181818183,"settings":33,"top_scores":0,"rank":23.0},{"model":"dnabert-3","mean_rank":23.454545454545453,"settings":33,"top_scores":0,"rank":24.0},{"model":"dnabert-5","mean_rank":23.727272727272727,"settings":33,"top_scores":1,"rank":25.0},{"model":"grover","mean_rank":24.0,"settings":33,"top_scores":1,"rank":26.0},{"model":"dnabert-4","mean_rank":24.606060606060606,"settings":33,"top_scores":0,"rank":27.0},{"model":"baseline_lstm","mean_rank":24.96969696969697,"settings":33,"top_scores":1,"rank":28.0},{"model":"dnabert-6","mean_rank":25.696969696969695,"settings":33,"top_scores":1,"rank":29.0},{"model":"hyena_large_1m","mean_rank":25.96969696969697,"settings":33,"top_scores":0,"rank":30.0},{"model":"gena-lm-bigbird-base-t2t","mean_rank":26.03030303030303,"settings":33,"top_scores":0,"rank":31.0},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean_rank":26.12121212121212,"settings":33,"top_scores":0,"rank":32.0},{"model":"phylogpn","mean_rank":26.90909090909091,"settings":33,"top_scores":0,"rank":33.0},{"model":"enformer","mean_rank":28.87878787878788,"settings":33,"top_scores":1,"rank":34.0},{"model":"dnabert2","mean_rank":29.060606060606062,"settings":33,"top_scores":0,"rank":35.0},{"model":"orthrus-4track","mean_rank":29.96969696969697,"settings":33,"top_scores":0,"rank":36.0},{"model":"gena-lm-bert-base-t2t","mean_rank":30.03030303030303,"settings":33,"top_scores":0,"rank":37.0},{"model":"cnn-56m-random-init","mean_rank":30.151515151515152,"settings":33,"top_scores":0,"rank":38.0},{"model":"gena-lm-bert-base-t2t-multi","mean_rank":31.454545454545453,"settings":33,"top_scores":0,"rank":39.0},{"model":"ntv2-50m-random-init","mean_rank":31.818181818181817,"settings":33,"top_scores":0,"rank":40.0},{"model":"gena-lm-bert-base","mean_rank":31.87878787878788,"settings":33,"top_scores":0,"rank":41.0},{"model":"rnabert","mean_rank":36.121212121212125,"settings":33,"top_scores":0,"rank":42.0},{"model":"baseline_cnn","mean_rank":36.666666666666664,"settings":33,"top_scores":0,"rank":43.0}],"Protein":[{"model":"generator_eukaryote_1_2b","mean_rank":5.133333333333334,"settings":15,"top_scores":3,"rank":1.0},{"model":"nucleotide-transformer-2.5b-multi-species","mean_rank":6.8,"settings":15,"top_scores":2,"rank":2.0},{"model":"ntv3-650m-pre","mean_rank":7.8,"settings":15,"top_scores":4,"rank":3.0},{"model":"nucleotide-transformer-v2-500m-multi-species","mean_rank":8.533333333333333,"settings":15,"top_scores":1,"rank":4.0},{"model":"ntv3-100m-pre","mean_rank":8.866666666666667,"settings":15,"top_scores":0,"rank":5.0},{"model":"nucleotide-transformer-2.5b-1000g","mean_rank":10.866666666666667,"settings":15,"top_scores":2,"rank":6.0},{"model":"ntv3-8m-pre","mean_rank":10.933333333333334,"settings":15,"top_scores":0,"rank":7.0},{"model":"nucleotide-transformer-v2-250m-multi-species","mean_rank":11.4,"settings":15,"top_scores":0,"rank":8.0},{"model":"nucleotide-transformer-v2-100m-multi-species","mean_rank":12.266666666666667,"settings":15,"top_scores":0,"rank":9.0},{"model":"nucleotide-transformer-v2-50m-multi-species","mean_rank":14.2,"settings":15,"top_scores":0,"rank":10.0},{"model":"rinalmo","mean_rank":14.666666666666666,"settings":15,"top_scores":0,"rank":11.0},{"model":"borzoi","mean_rank":15.133333333333333,"settings":15,"top_scores":1,"rank":12.0},{"model":"alphagenome","mean_rank":15.666666666666666,"settings":15,"top_scores":0,"rank":13.0},{"model":"nucleotide-transformer-500m-human-ref","mean_rank":15.8,"settings":15,"top_scores":0,"rank":14.0},{"model":"gena-lm-bert-large-t2t","mean_rank":17.2,"settings":15,"top_scores":0,"rank":15.0},{"model":"nucleotide-transformer-500m-1000g","mean_rank":17.4,"settings":15,"top_scores":0,"rank":16.0},{"model":"gena-lm-bert-base-lastln-t2t","mean_rank":19.733333333333334,"settings":15,"top_scores":0,"rank":17.0},{"model":"rnaernie","mean_rank":22.6,"settings":15,"top_scores":0,"rank":18.0},{"model":"grover","mean_rank":22.733333333333334,"settings":15,"top_scores":0,"rank":19.0},{"model":"evo2_1b_base","mean_rank":22.8,"settings":15,"top_scores":1,"rank":20.0},{"model":"splicebert","mean_rank":22.933333333333334,"settings":15,"top_scores":1,"rank":21.0},{"model":"phylogpn","mean_rank":23.0,"settings":15,"top_scores":0,"rank":22.0},{"model":"gena-lm-bigbird-base-t2t","mean_rank":23.2,"settings":15,"top_scores":0,"rank":23.0},{"model":"gena-lm-bert-base","mean_rank":24.066666666666666,"settings":15,"top_scores":0,"rank":24.0},{"model":"gena-lm-bert-base-t2t-multi","mean_rank":24.6,"settings":15,"top_scores":0,"rank":25.0},{"model":"ntv2-50m-random-init","mean_rank":25.866666666666667,"settings":15,"top_scores":0,"rank":26.0},{"model":"gena-lm-bert-base-t2t","mean_rank":26.066666666666666,"settings":15,"top_scores":0,"rank":27.0},{"model":"enformer","mean_rank":26.133333333333333,"settings":15,"top_scores":0,"rank":28.0},{"model":"baseline_cnn","mean_rank":28.2,"settings":15,"top_scores":0,"rank":29.0},{"model":"hyena_large_1m","mean_rank":28.8,"settings":15,"top_scores":0,"rank":30.0},{"model":"dnabert-6","mean_rank":29.2,"settings":15,"top_scores":0,"rank":31.0},{"model":"dnabert-3","mean_rank":29.333333333333332,"settings":15,"top_scores":0,"rank":32.0},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean_rank":29.6,"settings":15,"top_scores":0,"rank":33.0},{"model":"rnafm","mean_rank":29.8,"settings":15,"top_scores":0,"rank":34.0},{"model":"dnabert-5","mean_rank":30.133333333333333,"settings":15,"top_scores":0,"rank":35.0},{"model":"dnabert2","mean_rank":30.2,"settings":15,"top_scores":0,"rank":36.0},{"model":"hyena_medium_160k","mean_rank":30.4,"settings":15,"top_scores":0,"rank":37.0},{"model":"dnabert-4","mean_rank":30.666666666666668,"settings":15,"top_scores":0,"rank":38.0},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean_rank":31.133333333333333,"settings":15,"top_scores":0,"rank":39.0},{"model":"baseline_lstm","mean_rank":33.333333333333336,"settings":15,"top_scores":0,"rank":40.0},{"model":"orthrus-4track","mean_rank":33.93333333333333,"settings":15,"top_scores":0,"rank":41.0},{"model":"cnn-56m-random-init","mean_rank":36.333333333333336,"settings":15,"top_scores":0,"rank":42.0},{"model":"rnabert","mean_rank":38.53333333333333,"settings":15,"top_scores":0,"rank":43.0}]},"tasks":[{"id":"core_promoter_annotation","benchmark":"ours_classification","name":"Promoter Annotation","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9642505021278521,"sd":0.0019219885435732289,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"baseline_cnn","mean":0.8620879322715593,"sd":0.02066659299190426,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"baseline_lstm","mean":0.9487427346234467,"sd":0.0017642240746473673,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"borzoi","mean":0.9201748040092264,"sd":0.002117369615092802,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.9081719263211179,"sd":0.07698716366288526,"seeds":[1,3,5],"note":0,"rank":27.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9397194211538563,"sd":0.0029632640681072175,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9541349689448273,"sd":0.003458362872073182,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-3","mean":0.9624802210669061,"sd":0.00687055878584296,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-4","mean":0.9635596821153465,"sd":0.0021475483475542467,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert-5","mean":0.9677793480497272,"sd":0.0021544590734266485,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"dnabert-6","mean":0.964145337168841,"sd":0.002542837491740305,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert2","mean":0.7917161872085727,"sd":0.022021260913657976,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"enformer","mean":0.7463023781480913,"sd":0.003166517575189866,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"evo2_1b_base","mean":0.9567063171189235,"sd":0.0007601527006881458,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.8386929010192493,"sd":0.00796595356106948,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8902649389399884,"sd":0.001993458937815544,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.8273393316394648,"sd":0.016609664667628816,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.8526269085908638,"sd":0.005947503872854186,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.8902871054192266,"sd":0.0021667676516195786,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.8995115015247184,"sd":0.006094549066209283,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9515353152801578,"sd":0.006326906189455156,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"phylogpn","mean":0.7522337137249347,"sd":0.010808956422898922,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"grover","mean":0.8934119184911687,"sd":0.005807671500150503,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_large_1m","mean":0.9388992901065552,"sd":0.0031578120586001047,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.930665226945381,"sd":0.009081121849393655,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.7335874998813732,"sd":0.008412523281483897,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9511891734953349,"sd":0.009958255585774208,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9571071632090027,"sd":0.0004810691464554055,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.953949736826971,"sd":0.0027643867035896065,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9251009777132961,"sd":0.005119764800027369,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9432651362369631,"sd":0.0008650500697552766,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.9060852789960948,"sd":0.007120937354832808,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.8757109081637224,"sd":0.0011371773296927831,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9463041317212684,"sd":0.0013949762551222242,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9521518789408012,"sd":0.00466569335301794,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9573544642324936,"sd":0.004373401501880468,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.9424875877215144,"sd":0.000989556373600115,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.7591402528891756,"sd":0.008356640268508303,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rinalmo","mean":0.9628206238156108,"sd":0.0035874097391034765,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.8862507156550309,"sd":0.005031832368359839,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rnaernie","mean":0.9637707118574469,"sd":0.0005545779909181023,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rnafm","mean":0.9658435138555643,"sd":0.002833106092692169,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"splicebert","mean":0.9689645737582357,"sd":0.002332497479228697,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true}]},{"id":"domain","benchmark":"ours_classification","name":"Protein Domain Classification","modality":"Protein","metric":"ap_macro","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.6853196365720621,"sd":0.07910548973126563,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"baseline_cnn","mean":0.5472017832808272,"sd":0.010866851510592692,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"baseline_lstm","mean":0.5009075492456229,"sd":0.02183119566072667,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.6663245542312564,"sd":0.017066445018949387,"seeds":[1,2,3],"note":1,"rank":18.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5549601495165277,"sd":0.09445551841462349,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5634391313331819,"sd":0.02396021412469652,"seeds":[1,2,3],"note":1,"rank":30.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.30716366995599353,"sd":0.04277264529551656,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.5286000234922787,"sd":0.05405302838941573,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.5197400167665863,"sd":0.04219357242020316,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"dnabert-5","mean":0.4998924430917631,"sd":0.04972548842603861,"seeds":[1,2,3],"note":1,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.5302295379712261,"sd":0.03874921387375948,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false},{"model":"dnabert2","mean":0.7435309866491102,"sd":0.026162392415803336,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"enformer","mean":0.6063375506200367,"sd":0.019170581680115036,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false},{"model":"evo2_1b_base","mean":0.9076669113651047,"sd":0.06870256181920624,"seeds":[1,2,3],"note":2,"rank":4.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6120706893538843,"sd":0.04961661120496494,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7560728499201717,"sd":0.026728035880483038,"seeds":[1,2,3],"note":1,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5987043452268049,"sd":0.058181351224187224,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5968245181604213,"sd":0.02860968803718071,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7822893659796581,"sd":0.026334277474056755,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6188042066047219,"sd":0.08247389935975273,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9966827243018419,"sd":0.0029788509428794736,"seeds":[1,2,3],"note":1,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.5174920377892116,"sd":0.010640161950659515,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"grover","mean":0.6556255129509669,"sd":0.06391629414584866,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.4937543151222356,"sd":0.047915797767200105,"seeds":[1,2,3],"note":1,"rank":40.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5837133826942599,"sd":0.019547716249869635,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.47712923140310537,"sd":0.034832489839055326,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7849489051841688,"sd":0.010595387447334048,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9516921153165717,"sd":0.08367171067787615,"seeds":[1,2,3],"note":1,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.8868731181051772,"sd":0.051396158631129354,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7194164780118554,"sd":0.032010130607542105,"seeds":[1,2,3],"note":1,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8736162658796889,"sd":0.027604853195356535,"seeds":[1,2,3],"note":1,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6372979709665327,"sd":0.043734347133101266,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6660392611660279,"sd":0.01916590740336008,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7464657373523756,"sd":0.028025838621987965,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7337703218517637,"sd":0.0625409055290261,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9254058561220878,"sd":0.05932304262886514,"seeds":[1,4,5],"note":3,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7085406823054297,"sd":0.011936204893271634,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"orthrus-4track","mean":0.5337707871516563,"sd":0.013809942652196138,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"rinalmo","mean":0.8823047586236826,"sd":0.02103604033805152,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.32691739558643285,"sd":0.040029490058914964,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.6239006188292784,"sd":0.07761560311917415,"seeds":[1,2,3],"note":1,"rank":22.0,"top_score":false},{"model":"rnafm","mean":0.5870596962626119,"sd":0.04768825480949737,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"splicebert","mean":0.7339573547676418,"sd":0.026460195311517287,"seeds":[1,2,3],"note":1,"rank":13.0,"top_score":false}]},{"id":"enhancers_types","benchmark":"ours_classification","name":"Enhancer Types Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.716401361513767,"sd":0.009259855593647568,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.5353773273511879,"sd":0.0037414476086129124,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.5341774450839117,"sd":0.002069966943675698,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"borzoi","mean":0.6868280387616916,"sd":0.0006748036203638094,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5521683488618143,"sd":0.03093758356657798,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5623754360140842,"sd":0.001296926272537668,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5546165310376402,"sd":0.005190370376930165,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.5982469123643485,"sd":0.0032833073270506353,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-4","mean":0.5964050600579139,"sd":0.0065696855032414,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-5","mean":0.596134150655831,"sd":0.004730192650610727,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert-6","mean":0.5971259381210345,"sd":0.007667356074713501,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert2","mean":0.5777306098095071,"sd":0.005820253690490536,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"enformer","mean":0.6869198123541748,"sd":0.004934649740429211,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5496941262852891,"sd":0.004498118556173713,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5557767640927687,"sd":0.00209168594773786,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5827436575781447,"sd":0.006697902750625157,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5644021683840934,"sd":0.006062535562835696,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5636117949419732,"sd":0.0026674780799936806,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5769079133548111,"sd":0.004751739573625428,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5697750978869427,"sd":0.006043346291174969,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.624444245066293,"sd":0.015046352700067513,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.5725117110660428,"sd":0.005713411565130118,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"grover","mean":0.5600626134429113,"sd":0.005532329155156067,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5597099195155729,"sd":0.010474641419782858,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5477506354584495,"sd":0.005716676442287026,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5067810482865728,"sd":0.007132340491198108,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5661856000305614,"sd":0.0071331941658184135,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6103715053053782,"sd":0.005134982095370953,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.562832329639703,"sd":0.004328951930571847,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6015182221416068,"sd":0.005206311194378752,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6272297629999599,"sd":0.010220253734145349,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5628719634088588,"sd":0.0083590484391974,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5860393293573184,"sd":0.0042511766581026485,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.571771024984183,"sd":0.0013405356878603432,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6022315913265736,"sd":0.006908541681620315,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6150043665391394,"sd":0.008651492420409026,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5607133128080724,"sd":0.003666427188962587,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"orthrus-4track","mean":0.5340651585445686,"sd":0.0030657975681352666,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.5717735979605435,"sd":0.0076110997004796566,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rnabert","mean":0.5412389717034181,"sd":0.002046430149129279,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"rnaernie","mean":0.5935625340475673,"sd":0.0009076414742555891,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.5856139313865487,"sd":0.0008086279957075362,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"splicebert","mean":0.5995587475695648,"sd":0.007347572011845299,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false}]},{"id":"enzyme","benchmark":"ours_classification","name":"Enzyme Classification","modality":"Protein","metric":"ap_macro","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.35672683223286417,"sd":0.0192617990982904,"seeds":[1,2,3],"note":1,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.29852845149365104,"sd":0.01488919112655126,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.28757026897445137,"sd":0.002381988099415377,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"borzoi","mean":0.31514982789681323,"sd":0.011952330395594882,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.3118292985105789,"sd":0.007354885061733869,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3231068578995312,"sd":0.022702927386259338,"seeds":[1,2,3],"note":1,"rank":22.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.25356790545539876,"sd":0.015094814563899125,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.3036992363447185,"sd":0.01060086183240977,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"dnabert-4","mean":0.29821644403967706,"sd":0.0022644279760855607,"seeds":[1,2,3],"note":1,"rank":37.0,"top_score":false},{"model":"dnabert-5","mean":0.29485965413870613,"sd":0.01492534960072845,"seeds":[1,2,3],"note":1,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.3147503322222141,"sd":0.032636321310904846,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"dnabert2","mean":0.29666701134634466,"sd":0.03664279355188902,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"enformer","mean":0.29960413085210286,"sd":0.013068630843202308,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6882501857948992,"sd":0.041898110536984785,"seeds":[1,2,8],"note":4,"rank":1.0,"top_score":true},{"model":"gena-lm-bert-base","mean":0.31958565167404207,"sd":0.008019108289713209,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.328049105778563,"sd":0.007807098580679846,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.31791221032651645,"sd":0.028380222104629992,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.32309332138468455,"sd":0.055930886720110345,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3551906584380381,"sd":0.01854339213955071,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3163080167030267,"sd":0.024631203620999992,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5348124876061633,"sd":0.05189460355297917,"seeds":[1,2,3],"note":1,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.3513318033406701,"sd":0.007490504291034153,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"grover","mean":0.3330293284414292,"sd":0.015178676175795635,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"hyena_large_1m","mean":0.30235638424724914,"sd":0.04336696350576547,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.2938093961056776,"sd":0.025394950803687266,"seeds":[1,2,3],"note":1,"rank":40.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.3107074329041998,"sd":0.02836729833333196,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.40174873429737307,"sd":0.029204818530549206,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6116241522996675,"sd":0.1442661848186637,"seeds":[2,4,5],"note":5,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3885359062547225,"sd":0.027680387952489303,"seeds":[1,2,3],"note":1,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.409599924567313,"sd":0.011849875113066899,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.40440233556128397,"sd":0.018125799994443543,"seeds":[1,2,3],"note":1,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.31451105266605045,"sd":0.03063475058378572,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3376575461026087,"sd":0.02989122255206883,"seeds":[1,2,3],"note":1,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.38806105596671997,"sd":0.009059999919548832,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3935626977327815,"sd":0.018957349554978888,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.361904918193252,"sd":0.07612097000229176,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3434967592398254,"sd":0.021565043039300964,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"orthrus-4track","mean":0.31185729994225614,"sd":0.010222150251556154,"seeds":[1,2,3],"note":1,"rank":30.0,"top_score":false},{"model":"rinalmo","mean":0.41565575988291253,"sd":0.02573521432473293,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"rnabert","mean":0.2738063797392306,"sd":0.013501034386429996,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.33218485335769266,"sd":0.01386839727331308,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"rnafm","mean":0.3463019131390466,"sd":0.0026043729023303864,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"splicebert","mean":0.43120264681481874,"sd":0.07444903816199368,"seeds":[1,2,3],"note":1,"rank":4.0,"top_score":false}]},{"id":"noncodingrnafamily","benchmark":"ours_classification","name":"Non-coding RNA Classification","modality":"RNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9459007578424147,"sd":0.013256952769731482,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.7111842432922155,"sd":0.01190506917680288,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.8440604005261955,"sd":0.019233268798324255,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.9443985589731774,"sd":0.005955364953671374,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.9604016143221801,"sd":0.003969164246779918,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9742656046795871,"sd":0.008486412764394713,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9147957299995563,"sd":0.005987159364656647,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert-3","mean":0.9677305964646989,"sd":0.005053950677164124,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-4","mean":0.9647720477932573,"sd":0.0024394129020057216,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert-5","mean":0.9530959120083381,"sd":0.005671247542832252,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-6","mean":0.9658106189780009,"sd":0.004320260608821905,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert2","mean":0.8926405787996181,"sd":0.01578712041159405,"seeds":[1,3,4],"note":null,"rank":30.0,"top_score":false},{"model":"enformer","mean":0.7578805905051836,"sd":0.0026732008779240674,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"evo2_1b_base","mean":0.9439642361510739,"sd":0.009164922865692043,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.8964027874735314,"sd":0.016735885729387785,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.9247502614512991,"sd":0.010016299038277816,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.9452467766163188,"sd":0.005836496403168086,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.9368010978103783,"sd":0.006354274192753075,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9512332341147809,"sd":0.01089316396294845,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.9327892979029863,"sd":0.005907047204617239,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9234103386863308,"sd":0.00897129143778379,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"phylogpn","mean":0.6708003940061358,"sd":0.04731551971684854,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"grover","mean":0.9320568220558214,"sd":0.010495184509694385,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.8463995305008605,"sd":0.004901609064615177,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.8846068533649617,"sd":0.004184254109806405,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.7117336072986202,"sd":0.03040065034561905,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9377164612022564,"sd":0.009758196773224483,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9577057963340974,"sd":0.007867816968557307,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9062502903838477,"sd":0.0017144861150377995,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.945971280649533,"sd":0.005866812033599482,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9521194803629591,"sd":0.005786528908158935,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.87490578953799,"sd":0.01582665281123517,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.8531952641636036,"sd":0.00963974805220903,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.8634334618011463,"sd":0.012616808046683461,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.8935048751787045,"sd":0.01920707784884827,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9110144180947167,"sd":0.017030370921192174,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8300819886975455,"sd":0.0025860154109675466,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"orthrus-4track","mean":0.8300602581260061,"sd":0.01159842960494341,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.988542657925264,"sd":0.0021011075395081114,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"rnabert","mean":0.6818776321138039,"sd":0.01866817709403794,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.9706604289658998,"sd":0.001423546827290749,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnafm","mean":0.971168410032667,"sd":0.007714624435399096,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"splicebert","mean":0.9169147424967156,"sd":0.007639697478973507,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false}]},{"id":"ptm","benchmark":"ours_classification","name":"Post-translational Modifications Prediction","modality":"Protein","metric":"ap_macro","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.6712879103794317,"sd":0.003998620449537999,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"baseline_cnn","mean":0.5666540179762487,"sd":0.023217739198909945,"seeds":[1,2,3],"note":1,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.5688917404958113,"sd":0.014069757623970471,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.6333818012670762,"sd":0.026887911535891054,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6271665252260537,"sd":0.008653551750546174,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6759273805152368,"sd":0.011567072015579946,"seeds":[1,2,3],"note":1,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.3761502368410154,"sd":0.01734308978538907,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.5782708051506031,"sd":0.012305130365314183,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.592342138487658,"sd":0.013295247960049583,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.5786765581285134,"sd":0.023459549699657647,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"dnabert-6","mean":0.5954406837964146,"sd":0.01719209875384119,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"dnabert2","mean":0.6463841710122333,"sd":0.02333449977746392,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"enformer","mean":0.5764811211369393,"sd":0.008028206489736091,"seeds":[1,2,3],"note":1,"rank":37.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7003263043262978,"sd":0.0189952600231738,"seeds":[1,2,3],"note":2,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5959202882915401,"sd":0.03441714889343618,"seeds":[1,2,3],"note":1,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6561974538356303,"sd":0.012069612641023233,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6014781752232329,"sd":0.02171091657621739,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5803325772038072,"sd":0.014785836505932884,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6874479352041118,"sd":0.013177569140096263,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6656048249396461,"sd":0.026927464151458635,"seeds":[1,2,3],"note":1,"rank":18.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7437391497855811,"sd":0.006906906254901228,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.566797093293911,"sd":0.010311398780245777,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"grover","mean":0.6318422692811666,"sd":0.020520379996454854,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6382725544027332,"sd":0.014296924752972113,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6445838738470424,"sd":0.003178067677555232,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5812370692219776,"sd":0.026104408933843606,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7350015031785608,"sd":0.011855358472051977,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7578324960717595,"sd":0.004205622779158055,"seeds":[1,2,3],"note":1,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7442960700281447,"sd":0.0066219001404472155,"seeds":[1,2,3],"note":1,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6839451268192688,"sd":0.0073369283402177085,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7675740969739864,"sd":0.0015988571033712217,"seeds":[1,2,3],"note":1,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-1000g","mean":0.6647716225658165,"sd":0.0322172517392776,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6381151343145851,"sd":0.009797471930526559,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7231893645914304,"sd":0.017687105310821988,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7326179496287464,"sd":0.017552582465282688,"seeds":[1,2,3],"note":1,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7289294814638172,"sd":0.016816082581148924,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6689533511275766,"sd":0.02708570085439306,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"orthrus-4track","mean":0.5482170982810675,"sd":0.032308012427387865,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"rinalmo","mean":0.7586835435127783,"sd":0.019973859934034464,"seeds":[1,2,3],"note":1,"rank":2.0,"top_score":false},{"model":"rnabert","mean":0.49385076366334835,"sd":0.01685714579983326,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.6867545458256095,"sd":0.015546783072988771,"seeds":[1,2,3],"note":1,"rank":13.0,"top_score":false},{"model":"rnafm","mean":0.6461995456012289,"sd":0.011437383229170367,"seeds":[1,2,3],"note":1,"rank":22.0,"top_score":false},{"model":"splicebert","mean":0.6963720595126901,"sd":0.0036789173544566687,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false}]},{"id":"regulation","benchmark":"ours_classification","name":"Regulation Element Prediction","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.7966156391978905,"sd":0.0008636590024293765,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6507563546288289,"sd":0.0018854172516930058,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.6550367869909051,"sd":0.00416483710683469,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"borzoi","mean":0.7791469971775017,"sd":0.001716524995491256,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7099599008368513,"sd":0.004190978608514878,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7275780684196527,"sd":0.008485976420236213,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6617649243404794,"sd":0.009731480013428705,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-3","mean":0.6435072210121615,"sd":0.012236571346757208,"seeds":[1,2,4],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.6601647095370587,"sd":0.006177693346396661,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.6604219261313612,"sd":0.003657795068497778,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.658050629959876,"sd":0.007799225936727818,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert2","mean":0.6513830679206355,"sd":0.003152663681394728,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.7787435731797303,"sd":0.001183046928981291,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.705411150840347,"sd":0.008666891806570216,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6740382174569528,"sd":0.0022762229478723254,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7048910568355788,"sd":0.0024831372758966344,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6903887240587491,"sd":0.007191676096779478,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6963506478853395,"sd":0.0030190290682635713,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7249519999154866,"sd":0.0024441944120893117,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6947250862651425,"sd":0.003969467459073247,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7433508294485832,"sd":0.010208509289690582,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.6274019318109605,"sd":0.013217216872319878,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"grover","mean":0.6933206492885207,"sd":0.0036632366388017062,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6189538122283107,"sd":0.07543436757049232,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5268039888502123,"sd":0.055641619762293246,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6170158959226512,"sd":0.0077897210936019095,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7213226291164864,"sd":0.0035443976537994346,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7253067829511258,"sd":0.0038543991116387465,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7220838172086103,"sd":0.0028784358379459305,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7106017055333798,"sd":0.005005210233299841,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7342001582045259,"sd":0.008238574748500158,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6958684402832332,"sd":0.002195054197393503,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6877780337163488,"sd":0.005350018847233705,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7176412943216487,"sd":0.0008646487676934065,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7259772158031444,"sd":0.008498007851021321,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7477421792348418,"sd":0.004235750999972939,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7045690310184562,"sd":0.0032375607810602148,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.5158375428316685,"sd":0.03482662565421517,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.7350009326456386,"sd":0.004887901554992838,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.3531272109722636,"sd":0.001306703980476087,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7103988467224077,"sd":0.0032638755165882026,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"rnafm","mean":0.7053840026463256,"sd":0.00676816615657914,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.7218341579928638,"sd":0.004060817516445952,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false}]},{"id":"species_classification_1024","benchmark":"ours_classification","name":"Species Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5719810863318338,"sd":0.0044480395803749,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"baseline_cnn","mean":0.32916283746074315,"sd":0.004864456807763676,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.4910362742956388,"sd":0.006889418608008442,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"borzoi","mean":0.5349907730276174,"sd":0.006678876612879285,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5848621468423262,"sd":0.004042714330240092,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5660559471551412,"sd":0.010732359279160318,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.42595913931270657,"sd":0.021823895056978543,"seeds":[1,3,4],"note":0,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.4508872756138652,"sd":0.0032451646076662654,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.4517137170732977,"sd":0.008915179219814712,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-5","mean":0.43802261370921053,"sd":0.007829669764615155,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-6","mean":0.4392594663235994,"sd":0.0003299021965117339,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.6023630891399161,"sd":0.0017193556692676258,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"enformer","mean":0.5202382964018589,"sd":0.004860923147271402,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5248107228707334,"sd":0.014845709509883098,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.571004943737962,"sd":0.0047383582447918515,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5770795181549673,"sd":0.014798398285617229,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5689659307319483,"sd":0.006251614194674119,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5873772597353476,"sd":0.0053409503518145225,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.598711927711213,"sd":0.004760794175480016,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5700000880263716,"sd":0.0026526995281248105,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5994146124722045,"sd":0.0032896714815775583,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.5018923580792725,"sd":0.0004344524795662935,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"grover","mean":0.5490443728009577,"sd":0.00299029109397709,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3914568229329354,"sd":0.024466739296267647,"seeds":[1,4,5],"note":6,"rank":40.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4657476341565589,"sd":0.021239468344134017,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.38640149668848695,"sd":0.003095746160552147,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5615713736883841,"sd":0.008084941271748752,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5793412259061547,"sd":0.00449587306636596,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.539176062455415,"sd":0.007123060344001467,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5821788977505404,"sd":0.004331071833348588,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6028109892576884,"sd":0.008497750387587813,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5446855480954136,"sd":0.011592122866333219,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5532183399928343,"sd":0.0051441632271463636,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5803776863789789,"sd":0.012370555766994322,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5989733547458364,"sd":0.004876373801835705,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6065681138313725,"sd":0.001380422310681559,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5581554784021101,"sd":0.003944459107158284,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.4495418689934741,"sd":0.005517893966406665,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rinalmo","mean":0.577474895126508,"sd":0.001981656494361781,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"rnabert","mean":0.18449344257594708,"sd":0.0025048018071228144,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.5201509214629572,"sd":0.0017111208257593318,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rnafm","mean":0.549722630952208,"sd":0.004707608897327317,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.5777256481040495,"sd":0.006326377301953797,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false}]},{"id":"species_classification_128","benchmark":"ours_classification","name":"Species Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.30222599468678124,"sd":0.001316260313988516,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"baseline_cnn","mean":0.1866724167665086,"sd":0.001479905529770113,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.2514581324932279,"sd":0.00034948316806262083,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.27366946636514505,"sd":0.0021822654428400655,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.31959170607437454,"sd":0.0008814628016715511,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.26940856088651804,"sd":0.07200867043172933,"seeds":[1,6,7],"note":7,"rank":27.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.17272506214604066,"sd":0.00752158331515393,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.24264492662551607,"sd":0.07068016361199632,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.3034057723096879,"sd":0.009785399839135669,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-5","mean":0.2828444781407867,"sd":0.005707562781645496,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-6","mean":0.2955887496427406,"sd":0.005121395400748211,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert2","mean":0.32735616270712187,"sd":0.0039872910909215156,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"enformer","mean":0.2682074696052926,"sd":0.001961741822024064,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"evo2_1b_base","mean":0.20249387736112667,"sd":0.017003017090517193,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.2778035673495477,"sd":0.0034057782788993345,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.29711614503179895,"sd":0.007552419536452749,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.2638881959644212,"sd":0.002569905664987159,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.29469991314937133,"sd":0.005979899541125934,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.30877591328946685,"sd":0.00495032283592251,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2844241508007906,"sd":0.004150332202246598,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.3445677853291798,"sd":0.0022388442076364005,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.2560584104821905,"sd":0.001480563771214693,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"grover","mean":0.2786656416446858,"sd":0.002945926209470484,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"hyena_large_1m","mean":0.23958516766856267,"sd":0.012361333925118621,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.2560899800319104,"sd":0.0016250257164562346,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2122965293572362,"sd":0.0030449414669581716,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.2920271571336883,"sd":0.0025188209536959397,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.2999592298979219,"sd":0.002538687175615646,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.27311784363161407,"sd":0.0032378496390294506,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2867771413606541,"sd":0.0028932402311099385,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.33065467396819465,"sd":0.0017065358520673148,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.2640397698911263,"sd":0.006174039192405012,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.258314575645937,"sd":0.006187287130302117,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.30082241245759433,"sd":0.006710445175493457,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.31116061706613474,"sd":0.0014982884617035946,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.3320095451616146,"sd":0.004345142258947848,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.27128170832107174,"sd":0.0027221465040939826,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"orthrus-4track","mean":0.2342785535775849,"sd":0.0026665462871704147,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.3125380632912366,"sd":0.0028141202144797957,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.18852705939305642,"sd":0.0024152130863577554,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.25741415484760827,"sd":0.002530616884339381,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rnafm","mean":0.292878059358992,"sd":0.0026004716886052696,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"splicebert","mean":0.3256235834661026,"sd":0.0028625180336647855,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false}]},{"id":"species_classification_2048","benchmark":"ours_classification","name":"Species Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.685362994739266,"sd":0.006286901570073486,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"baseline_cnn","mean":0.392325430104328,"sd":0.005030322289282845,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.6194660588138071,"sd":0.008093898926764324,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"borzoi","mean":0.6605820742542503,"sd":0.007134109783858558,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6796164605471066,"sd":0.03497184470790073,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6843360618991574,"sd":0.0039148122605389924,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.18223566431860366,"sd":0.027352326131254105,"seeds":[3,4,5],"note":8,"rank":42.0,"top_score":false},{"model":"dnabert-3","mean":0.4606307190912805,"sd":0.001364726166359113,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-4","mean":0.46251263367300255,"sd":0.006282984715287261,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-5","mean":0.4431592063077601,"sd":0.017718019430513068,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-6","mean":0.45554366521739587,"sd":0.003443688368239718,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert2","mean":0.6974262947146358,"sd":0.001788887627134945,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"enformer","mean":0.6494423226905531,"sd":0.01096352921907671,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6522887263747202,"sd":0.007696946712669299,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.676659441992425,"sd":0.009442422532053336,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6883308503216503,"sd":0.0038704988062254705,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6762223813123994,"sd":0.00509662905358888,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6922166781046095,"sd":0.01172985173421332,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6992209528641257,"sd":0.0007116137476314233,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6820728952491482,"sd":0.006302449933283673,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7062984511943515,"sd":0.003573417809679868,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.6080781148060987,"sd":0.00906681541806818,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.6624290661482506,"sd":0.005816808370515206,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5924056040037223,"sd":0.03548000178534307,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5813695899034447,"sd":0.004494229816880233,"seeds":[1,4,7],"note":9,"rank":32.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.49536715595889086,"sd":0.012427608699506114,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6717224201101746,"sd":0.003147250762532343,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6952014560742583,"sd":0.0026498177223010185,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6635271599596078,"sd":0.00149455977566894,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6878097328731527,"sd":0.009361978185943745,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.713069014728012,"sd":0.005251371878782831,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-1000g","mean":0.6518962656024304,"sd":0.0058878398764586875,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6604917260622059,"sd":0.0015051497653411664,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.68732849627925,"sd":0.005087502924947228,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7007406447579578,"sd":0.0020691291015948474,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7088729978205748,"sd":0.00047148688687919,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6759064297365407,"sd":0.0024514626071430582,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":0.5614127229488511,"sd":0.002817792148256765,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.6832058607651811,"sd":0.008028351229085063,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"rnabert","mean":0.17936456348239566,"sd":0.00510836620990629,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.6252843490239869,"sd":0.0018332314799590522,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnafm","mean":0.5409510376821963,"sd":0.008972052812033672,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"splicebert","mean":0.5690493289255525,"sd":0.002003711490839545,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false}]},{"id":"species_classification_256","benchmark":"ours_classification","name":"Species Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.36112195460012175,"sd":0.0026069965067493216,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"baseline_cnn","mean":0.22057190120271572,"sd":0.005422214766031464,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.3023350698238205,"sd":0.004930224050491257,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.32717896482531333,"sd":0.003233472417796918,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.35747252544375724,"sd":0.006073830762090114,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3042804083424262,"sd":0.06235781730787175,"seeds":[1,2,6],"note":10,"rank":34.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.22917014942024577,"sd":0.027264276707380908,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.36520060365447954,"sd":0.003208972177141394,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-4","mean":0.37073750465231253,"sd":0.0052787104359647875,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert-5","mean":0.35402353691766325,"sd":0.0025080455544960263,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"dnabert-6","mean":0.35009805952820344,"sd":0.0029585984537341225,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert2","mean":0.39924571140811144,"sd":0.005873252991652457,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"enformer","mean":0.3221467065130742,"sd":0.00019607590932030578,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"evo2_1b_base","mean":0.24938486969414994,"sd":0.036423058408982176,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.35088555194979515,"sd":0.004194638023467139,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.36868625483592676,"sd":0.008501970228717407,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3490460191282803,"sd":0.013307761784538394,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.38261762348226297,"sd":0.0032657111704219996,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3916246771370822,"sd":0.009073968796744422,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.36711117518537195,"sd":0.006998921447022661,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.41578426756868303,"sd":0.0052209681614093445,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.30775904238943114,"sd":0.008530387792457371,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.3439265987541819,"sd":0.00491507073126742,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_large_1m","mean":0.28273385149529595,"sd":0.022359744245698157,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3015140164774048,"sd":0.006120892261382846,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2374277410200471,"sd":0.002303746900642419,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.35425755972128775,"sd":0.001591238969159514,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.3769195690395942,"sd":0.0021725675392734833,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3362561977740956,"sd":0.004125038421614093,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.3681649959554199,"sd":0.004811611541116526,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.4029084293470272,"sd":0.007311228744808496,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.33416503938715125,"sd":0.004752144155600576,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3238672039209791,"sd":0.0066463088606981725,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3722362024499303,"sd":0.0036024604357933186,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3923310887841026,"sd":0.004548717255374249,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.4021109751573629,"sd":0.00748308880905581,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3330394226050789,"sd":0.010128007140129671,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"orthrus-4track","mean":0.27335276725750146,"sd":0.0047368520563851515,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.37827938054184695,"sd":0.008364861945973585,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.18875958951739885,"sd":0.0016341719411988088,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.3267656062401636,"sd":0.0035424521878105074,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rnafm","mean":0.34949978395007003,"sd":0.004393940038944894,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.38771235283189087,"sd":0.0024720005485803706,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false}]},{"id":"species_classification_512","benchmark":"ours_classification","name":"Species Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.45162350938256546,"sd":0.0015189623497595289,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"baseline_cnn","mean":0.2604633859714986,"sd":0.0019125748654203007,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.3796574212200742,"sd":0.002752054789843092,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.4197153886658086,"sd":0.0038927973877320187,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4874832294396543,"sd":0.005128509036301898,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.4676960284562581,"sd":0.009074563437709912,"seeds":[1,4,5],"note":6,"rank":12.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.31997892308504966,"sd":0.026740614695703595,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.4589016658055233,"sd":0.006540843421165447,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-4","mean":0.4516742088659277,"sd":0.005361949902010778,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert-5","mean":0.4391724760051782,"sd":0.010950365330433408,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-6","mean":0.4284449932284802,"sd":0.0017487102244140838,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert2","mean":0.49129081734349156,"sd":0.005398489325911548,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"enformer","mean":0.39843646775124314,"sd":0.002138086600079268,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3895305396441983,"sd":0.01020392852898969,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4375571485609336,"sd":0.003440406111161948,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.4620609363899984,"sd":0.013995376581135381,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.45446452774881785,"sd":0.007892963639096381,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.4821723847490688,"sd":0.0019475513319624957,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.48406668226103317,"sd":0.004164591581438552,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4635153756497633,"sd":0.005481129751159639,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.49565115026556955,"sd":0.0026449656376108365,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.39871071112658396,"sd":0.0012068562205946349,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.4336011522493473,"sd":0.012245284711298418,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3377961421898992,"sd":0.033490299211740425,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.36353755524448755,"sd":0.021399152000918034,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2810193443771023,"sd":0.0026987451521405044,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.4432304448690855,"sd":0.002888500805476065,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.4686627059988246,"sd":0.012598461399570543,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.4193187596975098,"sd":0.009192748856689895,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.4676663894045627,"sd":0.0012232337142034939,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.49409895920772817,"sd":0.0019462031540340712,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.42205898429026617,"sd":0.004853082412823762,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.42632201425933897,"sd":0.003603402238645786,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.47037239824484023,"sd":0.004492103806938888,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.48762788122101713,"sd":0.004138714744242746,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.49958987465131555,"sd":0.005286664771424108,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.427065751001411,"sd":0.01055008256890146,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"orthrus-4track","mean":0.3441898755405092,"sd":0.004194203005589819,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.46644485959693394,"sd":0.0033648234682668856,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"rnabert","mean":0.18965741099163433,"sd":0.0021430937301283827,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.4033234458041723,"sd":0.007047750587918844,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rnafm","mean":0.42681678564926995,"sd":0.0029867684147369985,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"splicebert","mean":0.47409462173122113,"sd":0.0014552151636052846,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false}]},{"id":"splice_sites_acceptor","benchmark":"ours_classification","name":"Splice Sites Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9917921770740757,"sd":0.0002485097912242035,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.9505096414581954,"sd":0.0007498957129016955,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.9434806642918707,"sd":0.004238488840235615,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.9772756471958425,"sd":0.001122168225192199,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8647339965523314,"sd":0.00486084320192696,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9106649589790784,"sd":0.006672261236020183,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9799133523996899,"sd":0.0006009038447115384,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-3","mean":0.9820590871773914,"sd":0.006343690759187056,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert-4","mean":0.9848124299121642,"sd":0.0003562044507620293,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-5","mean":0.9854000071122567,"sd":0.0009039275985296448,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-6","mean":0.9839273409547116,"sd":0.0009332192943053201,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert2","mean":0.9121690913493845,"sd":0.0012737452299719728,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"enformer","mean":0.9682095738354342,"sd":0.000656162140622286,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"evo2_1b_base","mean":0.9844459410403555,"sd":0.0030278775173581585,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.9307837398985199,"sd":0.0033208992514178796,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.9460983668080002,"sd":0.0046902849729661845,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.9510423964300917,"sd":0.00041108349194333895,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.9442220165910942,"sd":0.002679183754553641,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9593387475693547,"sd":0.000984120726402062,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.9573389072155476,"sd":0.003035874103642799,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9883069234401121,"sd":0.0002821520316351359,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.9551475716621013,"sd":0.0008270351103871733,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.9536287881823757,"sd":0.0007546087369915532,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"hyena_large_1m","mean":0.9551431001418403,"sd":0.010115045793748417,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.9554930774964161,"sd":0.012773130342952996,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.9219088883136821,"sd":0.009576931082646562,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9847417176246347,"sd":0.0011074151029936705,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9834090783711086,"sd":0.001749896519573018,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9848776587040718,"sd":0.000757066682939914,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9853749368118618,"sd":0.001121203991715481,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9879661004635977,"sd":0.0007579498983457934,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.9793167892449615,"sd":0.0013159559047539338,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.9766743462578746,"sd":0.001564104435626424,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9874532170487468,"sd":0.0007322734733991775,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9882044714315157,"sd":0.00041020893568033927,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9895804515045125,"sd":0.0005061131661653447,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.9853199664472027,"sd":0.0004370527385030237,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.7459554505281956,"sd":0.00818017968505192,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rinalmo","mean":0.9897083687506557,"sd":0.0010498924820132665,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"rnabert","mean":0.9654564096659305,"sd":0.0011718187368620146,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnaernie","mean":0.9831035835935453,"sd":0.0009126016100107142,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"rnafm","mean":0.9811022601615241,"sd":0.0005328151058729851,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.990353789770756,"sd":0.0003212239047088885,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false}]},{"id":"splice_sites_donor","benchmark":"ours_classification","name":"Splice Sites Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9925714728568578,"sd":0.0004989939640221915,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.9582990888945061,"sd":0.0014719118692449018,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"baseline_lstm","mean":0.9480363233907011,"sd":0.0021248229623605686,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.9796192477928717,"sd":0.0002465326492121274,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.9016342328228384,"sd":0.006595808972935248,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9317434582048786,"sd":0.05543718423753529,"seeds":[1,3,4],"note":6,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9869555562593776,"sd":0.0010481695720731042,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-3","mean":0.9880754754002309,"sd":0.0009767980228868984,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-4","mean":0.9889299576572761,"sd":0.0007038918379574538,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-5","mean":0.9871660459653976,"sd":0.001595320495998873,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert-6","mean":0.9892699901608211,"sd":0.0007991452820414453,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert2","mean":0.9106001030218377,"sd":0.0063683826107585354,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.9689763829394726,"sd":0.0003902181395409729,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"evo2_1b_base","mean":0.985219023452085,"sd":0.00015038593783289124,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.9295261652841064,"sd":0.004134576668566397,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.9500067745955864,"sd":0.0010379093817847074,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.9497698224380982,"sd":0.004150661522298171,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.9531133684290307,"sd":0.0009955209100549456,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9638923214882282,"sd":0.0011373887913993903,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.9633266091669731,"sd":0.0008955430308479549,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9885195424494327,"sd":0.0017048665444093316,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"phylogpn","mean":0.9570695239351398,"sd":0.0009786361657634934,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"grover","mean":0.9579634503861533,"sd":0.002592518092846343,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_large_1m","mean":0.9467908009358794,"sd":0.012458744248932592,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.9671304772570695,"sd":0.0049442994619058895,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.9334723021594579,"sd":0.0074583261233826074,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9864842904730231,"sd":0.0010459555199198666,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9872839551604039,"sd":0.0014417114045636398,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.984551502084415,"sd":0.001691656150605932,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.986550730599857,"sd":0.0006525614635773253,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.990884355064829,"sd":0.0004165125386137473,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.979760375650904,"sd":0.0010745862121135273,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.9782553310999142,"sd":0.0008450073877619099,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9891389748775007,"sd":0.00045653268188018974,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9897935892325033,"sd":0.001677766465796753,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9909240410668089,"sd":0.0005550616033194318,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.9875438194096237,"sd":0.0005772889165603287,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"orthrus-4track","mean":0.7855134774696342,"sd":0.013845385305039723,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rinalmo","mean":0.9908473889268177,"sd":0.0006711428940827617,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"rnabert","mean":0.9625875950000771,"sd":0.00039606617333042216,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rnaernie","mean":0.9878553627611826,"sd":0.001231751551551751,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"rnafm","mean":0.9866240929153747,"sd":0.0010209857505414724,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"splicebert","mean":0.9914176302969443,"sd":0.00031823586345997284,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false}]},{"id":"structure","benchmark":"ours_classification","name":"Secondary Structure Analysis","modality":"Protein","metric":"ap_macro","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.40429422522140396,"sd":0.01845190089086192,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"baseline_cnn","mean":0.37106454638015174,"sd":0.01267472988985876,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.4096655467254743,"sd":0.014821699175967724,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.4385482920200317,"sd":0.0143289059554096,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4511830821819082,"sd":0.0274239154774944,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.38901501628553853,"sd":0.012512432389516433,"seeds":[1,2,3],"note":1,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.32131648599323864,"sd":0.006840141236288738,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.4634838794962364,"sd":0.024330656805355576,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"dnabert-4","mean":0.43793746371901404,"sd":0.03982664115533725,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"dnabert-5","mean":0.4061468502125705,"sd":0.034194066131335465,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.39702294568904123,"sd":0.00237047671228507,"seeds":[1,2,3],"note":1,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.5289534080977115,"sd":0.026616140494157024,"seeds":[1,2,3],"note":1,"rank":13.0,"top_score":false},{"model":"enformer","mean":0.39772640852409075,"sd":0.01648750056598617,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6673868830681694,"sd":0.034120816181726334,"seeds":[1,2,3],"note":2,"rank":7.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.45410841347181524,"sd":0.039334593016213615,"seeds":[1,2,3],"note":1,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.44154364987259054,"sd":0.02885102383603251,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.43783145653852246,"sd":0.023270512096087104,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.44647285669628284,"sd":0.024988270552413774,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.4732161351531232,"sd":0.026978378028599315,"seeds":[1,2,3],"note":1,"rank":15.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4206385445436922,"sd":0.037671235891068126,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6564148938120928,"sd":0.02449066674308007,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"phylogpn","mean":0.4305898505324253,"sd":0.04555142923163161,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.4712352181007839,"sd":0.047257975857636865,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"hyena_large_1m","mean":0.4458115652985533,"sd":0.05553521572408942,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.37370400689883554,"sd":0.02678490300669348,"seeds":[1,2,3],"note":1,"rank":40.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.42297943740267413,"sd":0.05776127712947966,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6696847639370009,"sd":0.02409648637512274,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7352678281934989,"sd":0.015420876090439975,"seeds":[1,2,3],"note":1,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7154200646237266,"sd":0.00750696543591055,"seeds":[1,2,3],"note":1,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5325486806259117,"sd":0.004946704336184803,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6591932222910644,"sd":0.03946906789375148,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.4306584989577912,"sd":0.011780004376099497,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4440919217809076,"sd":0.021429446215245306,"seeds":[1,2,3],"note":1,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5735736933243136,"sd":0.031729520919141746,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6821733596438911,"sd":0.03181846657003823,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7482687583443772,"sd":0.015302450876977323,"seeds":[1,3,4],"note":5,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5098245248998077,"sd":0.015839320726331162,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"orthrus-4track","mean":0.39294236424113316,"sd":0.020732276736093187,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.7090228528757748,"sd":0.022340818547109453,"seeds":[1,2,3],"note":1,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.33131208330018147,"sd":0.01975655234259583,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.43888152729232754,"sd":0.015406581068047752,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.4314747844319866,"sd":0.0061979652172378125,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"splicebert","mean":0.6430984075078263,"sd":0.010607009631166132,"seeds":[1,2,3],"note":1,"rank":10.0,"top_score":false}]},{"id":"transmembrane","benchmark":"ours_classification","name":"Transmembrane Region Prediction","modality":"Protein","metric":"ap_macro","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.5391182266145151,"sd":0.05448830381544796,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"baseline_cnn","mean":0.4006751294035933,"sd":0.03975360211770053,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.36931985039750437,"sd":0.008261125628440759,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.5193285362859034,"sd":0.021205586060354706,"seeds":[1,2,3],"note":1,"rank":18.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.45045170381643673,"sd":0.0484201993154086,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.4108701616562623,"sd":0.017310870494775935,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.40240699512801814,"sd":0.027467611357095155,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"dnabert-3","mean":0.35250660792290855,"sd":0.006665292446618359,"seeds":[1,2,3],"note":1,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.32606887270694335,"sd":0.006943276769456423,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"dnabert-5","mean":0.3263553052438008,"sd":0.020637210235240126,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"dnabert-6","mean":0.34888072338925696,"sd":0.03247913552985641,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"dnabert2","mean":0.36216655846150325,"sd":0.07168088324966357,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"enformer","mean":0.4205657273139368,"sd":0.00514196211921282,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5279624991996551,"sd":0.06699267942155611,"seeds":[1,2,3],"note":2,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5114478068875707,"sd":0.016597766708740982,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5166840762855841,"sd":0.0045498315832989125,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.494428402761955,"sd":0.0638780559802398,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5206542292911723,"sd":0.023146953091483943,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.47706121349754854,"sd":0.013336931079827656,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5279521812433166,"sd":0.044842742689935204,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7858722317131972,"sd":0.0314601022356383,"seeds":[1,2,3],"note":1,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.33439364194276067,"sd":0.03139774792407005,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"grover","mean":0.48439283939891215,"sd":0.00891929566650201,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5175424747285305,"sd":0.007882685055278352,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.49288523047943505,"sd":0.04177421189159887,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.4407283218718495,"sd":0.043851432812985286,"seeds":[1,2,3],"note":1,"rank":30.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6935650783683022,"sd":0.027820021891978144,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7942208836752139,"sd":0.08518568189606429,"seeds":[1,2,3],"note":1,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.7153763327885914,"sd":0.04417429478760148,"seeds":[1,2,3],"note":1,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6202393181672784,"sd":0.024005442021187363,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7053676784978072,"sd":0.03155001246392486,"seeds":[1,2,3],"note":1,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5194326556028608,"sd":0.04138873052157277,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4970063134203985,"sd":0.004285099142941634,"seeds":[1,2,3],"note":1,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6144831853872457,"sd":0.015499981349319094,"seeds":[1,2,3],"note":1,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6976941431323632,"sd":0.05043693653824345,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6907627259927182,"sd":0.016017698307376024,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5807478830510823,"sd":0.020184561500571775,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.3356853893035983,"sd":0.013173531092459945,"seeds":[1,2,3],"note":1,"rank":39.0,"top_score":false},{"model":"rinalmo","mean":0.6927580107938934,"sd":0.012758278807621594,"seeds":[1,2,3],"note":1,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.24907624033325834,"sd":0.004029409892528063,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.525699859274728,"sd":0.006274379967979269,"seeds":[1,2,3],"note":1,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.47502376780792116,"sd":0.022624008435970875,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"splicebert","mean":0.48581503322809233,"sd":0.0114665740874372,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false}]},{"id":"variant","benchmark":"ours_classification","name":"Protein Variant Classification","modality":"Protein","metric":"ap_binary","protocol_variant":"full_source_model_caps_v1","results":[{"model":"alphagenome","mean":0.34686714652231543,"sd":0.006161596625751005,"seeds":[1,2,3],"note":1,"rank":8.0,"top_score":false},{"model":"baseline_cnn","mean":0.31070075270840025,"sd":0.010556043079496834,"seeds":[1,2,3],"note":1,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.2889836341532262,"sd":0.0023243583515628647,"seeds":[1,2,3],"note":1,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.32597892706710013,"sd":0.011202441602663921,"seeds":[1,2,3],"note":1,"rank":24.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.34365622783242145,"sd":0.006518824753365753,"seeds":[1,2,3],"note":1,"rank":10.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.338353289819393,"sd":0.01084588736558946,"seeds":[1,2,3],"note":1,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.2819591425068527,"sd":0.015347053857449037,"seeds":[1,2,3],"note":1,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.29161581382386337,"sd":0.0030902988445841817,"seeds":[1,2,3],"note":1,"rank":38.0,"top_score":false},{"model":"dnabert-4","mean":0.28662524718101307,"sd":0.004620231531597487,"seeds":[1,2,3],"note":1,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.29531153879887023,"sd":0.0008800669843292892,"seeds":[1,2,3],"note":1,"rank":37.0,"top_score":false},{"model":"dnabert-6","mean":0.3160756369436559,"sd":0.014688392033933685,"seeds":[1,2,3],"note":1,"rank":31.0,"top_score":false},{"model":"dnabert2","mean":0.305321580810342,"sd":0.010929883855310811,"seeds":[1,2,3],"note":1,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.32312032869477136,"sd":0.008095565732033806,"seeds":[1,2,3],"note":1,"rank":27.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3361109101338798,"sd":0.01407339582960065,"seeds":[1,2,3],"note":2,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.31871937394370853,"sd":0.016629758242631073,"seeds":[1,2,3],"note":1,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.3415928966547939,"sd":0.002052958257845654,"seeds":[1,2,3],"note":1,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.32232186567860227,"sd":0.007585686546760438,"seeds":[1,2,3],"note":1,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.3245959566699633,"sd":0.02702137086411128,"seeds":[1,2,3],"note":1,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3366935381409997,"sd":0.011919120868032079,"seeds":[1,2,3],"note":1,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.34446199078154643,"sd":0.004190590130746856,"seeds":[1,2,3],"note":1,"rank":9.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.36410412505673684,"sd":0.03341140900118387,"seeds":[1,2,3],"note":1,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.33389268108991005,"sd":0.0017787422825756136,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"grover","mean":0.3318691767794354,"sd":0.007736007041316295,"seeds":[1,2,3],"note":1,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.35803920698020203,"sd":0.001868515857721813,"seeds":[1,2,3],"note":1,"rank":3.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.35117823392275005,"sd":0.0022965221758687827,"seeds":[1,2,3],"note":1,"rank":5.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.3010683874167281,"sd":0.018226784561963614,"seeds":[1,2,3],"note":1,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.3398898496028508,"sd":0.012070817108983462,"seeds":[1,2,3],"note":1,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.3738313984808054,"sd":0.03581006108266271,"seeds":[1,2,3],"note":1,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.3359162019387629,"sd":0.008783213688087245,"seeds":[1,2,3],"note":1,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.3343505186128815,"sd":0.012518168485801492,"seeds":[1,2,3],"note":1,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.350017055650093,"sd":0.005068246812060932,"seeds":[1,2,3],"note":1,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.33701276014293885,"sd":0.005842372223686959,"seeds":[1,2,3],"note":1,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3537396358383764,"sd":0.0065540863642181485,"seeds":[1,2,3],"note":1,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3476535132583472,"sd":0.008341827914927482,"seeds":[1,2,3],"note":1,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.341625527036878,"sd":0.0012227946024599002,"seeds":[1,2,3],"note":1,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.34317160036731326,"sd":0.0018055889999472599,"seeds":[1,2,3],"note":1,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3251029436930025,"sd":0.012756284394772666,"seeds":[1,2,3],"note":1,"rank":25.0,"top_score":false},{"model":"orthrus-4track","mean":0.32276256889173105,"sd":0.016631420702622378,"seeds":[1,2,3],"note":1,"rank":28.0,"top_score":false},{"model":"rinalmo","mean":0.27800332153270224,"sd":0.021016099590075936,"seeds":[1,2,3],"note":1,"rank":43.0,"top_score":false},{"model":"rnabert","mean":0.2783377774615893,"sd":0.008564017715743498,"seeds":[1,2,3],"note":1,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.3339694204220979,"sd":0.004903306131274572,"seeds":[1,2,3],"note":1,"rank":21.0,"top_score":false},{"model":"rnafm","mean":0.3109596482080334,"sd":0.0035679360886223884,"seeds":[1,2,3],"note":1,"rank":32.0,"top_score":false},{"model":"splicebert","mean":0.30847085959567766,"sd":0.0059118776041430345,"seeds":[1,2,3],"note":1,"rank":34.0,"top_score":false}]},{"id":"cpg_methylation","benchmark":"ours_multilabel_classification","name":"CpG Methylation Prediction (Classification)","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9907897877213309,"sd":0.000515661261314487,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.9730609747023269,"sd":0.0007787125596767835,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"baseline_lstm","mean":0.9775784036824557,"sd":0.0011571061524704046,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.9880075446478139,"sd":0.0003307650002010851,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.9829616583332516,"sd":0.002746280712448758,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9860611546590792,"sd":0.0008120737396636933,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9804096486973152,"sd":0.0019227657339349927,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.9883938786641286,"sd":0.0005587601312934195,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"dnabert-4","mean":0.9869284374519613,"sd":0.0004927196784618544,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-5","mean":0.9875904787273645,"sd":0.00160329013101677,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-6","mean":0.9868608603999645,"sd":0.0003257781572565296,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert2","mean":0.9761363186370192,"sd":0.0027844080790177888,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"enformer","mean":0.9840042786339743,"sd":0.0006944154561225936,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"evo2_1b_base","mean":0.9841024468099383,"sd":0.0019882328394105362,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.9820883992709109,"sd":0.0010030303004673763,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.9831123382453265,"sd":0.0004559420099145873,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.9837088781612322,"sd":0.0008169847838883198,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.9835528777095718,"sd":0.0006410962043488687,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9852866080105968,"sd":0.0006041447187414745,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.9809114527161948,"sd":0.0009541753462613291,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9873905362141323,"sd":0.0004920931618002884,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"phylogpn","mean":0.9776337963653441,"sd":0.00031752921077669584,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"grover","mean":0.9850958970469887,"sd":0.00036004578023325666,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":0.9725650632418161,"sd":0.0009259808310320885,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.9819116176389306,"sd":0.0015520076569279704,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.968427492474606,"sd":0.00062835078566893,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9845523943929114,"sd":0.002238838459302981,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9885651188159041,"sd":0.00028699856601201125,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.984776779379643,"sd":0.0005256859770873321,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9852060575460282,"sd":0.0007375535927486099,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9878544943141722,"sd":0.0005218284143959637,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.9822334340674699,"sd":0.0003402405442771416,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.9813302140639422,"sd":0.001152561939544588,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9818741476986023,"sd":0.0028788495830978492,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9832801468891319,"sd":0.0010409822036524779,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9855942720953451,"sd":0.002115290810217912,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.9841327453886112,"sd":0.0018414027890344277,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.9645405711878767,"sd":0.0016175533872193747,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.9874801179069003,"sd":0.000882518130367991,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.9098952920030188,"sd":0.024511799850525453,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.9852358881318425,"sd":0.0006698954445294824,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"rnafm","mean":0.9854677611178411,"sd":0.0001167614389348425,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"splicebert","mean":0.9866762940180683,"sd":0.0016862412443271236,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false}]},{"id":"histone_annotation_1","benchmark":"ours_multilabel_classification","name":"Epigenetic Marks Prediction (Classification)","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5406784613949623,"sd":0.0024148993229629405,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.33747159671513827,"sd":0.0013353725763414947,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.3884868175962032,"sd":0.0008912878551267716,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.5189105833114649,"sd":0.0008445901171646817,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.43988167390331284,"sd":0.004513394811863794,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.4175196477974946,"sd":0.06334482410095367,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.366596093980737,"sd":0.0032718441033864495,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.311245208817278,"sd":0.10996292893493116,"seeds":[1,3,4],"note":0,"rank":42.0,"top_score":false},{"model":"dnabert-4","mean":0.38091124756056444,"sd":0.0022037024408020204,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-5","mean":0.3821221597386501,"sd":0.004202846623446693,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-6","mean":0.3827861207576921,"sd":0.0030527107316090756,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert2","mean":0.4377250311529286,"sd":0.002096877676310656,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"enformer","mean":0.5202580708631669,"sd":0.0004259638640269231,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"evo2_1b_base","mean":0.4103575728446309,"sd":0.008719708575671909,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.42530022603182843,"sd":0.001953519830328741,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.45149503522665146,"sd":0.0025737789664150184,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.4408760876821175,"sd":0.0023994329365288653,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.435695344374687,"sd":0.004761797558912247,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.46549707585027006,"sd":0.009981606268548278,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4497997409201106,"sd":0.004539084009190712,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.4716663700516575,"sd":0.0012695476141321076,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.41878203657965885,"sd":0.002381536558862938,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"grover","mean":0.4270347195921994,"sd":0.0017018926841798827,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3881337957868815,"sd":0.001038215659587472,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4153600847584753,"sd":0.013366908493992453,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.34964474253730876,"sd":0.0031647273367725157,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.4390690937784496,"sd":0.0068181969663968596,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.4611206583202313,"sd":0.0030131491706679426,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.4253490950486288,"sd":0.00268178299030541,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.4509581071967974,"sd":0.0019423442971763457,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.47667657296186045,"sd":0.0015059532100751645,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.41599022522912504,"sd":0.001877436536181545,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.41591458804977244,"sd":0.0016129756168913455,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.4480442333232286,"sd":0.005187740438797782,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.47121506043671013,"sd":0.00356464584815697,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.4714034823000751,"sd":0.001283564843931401,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.4252422625535595,"sd":0.002446144983651963,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"orthrus-4track","mean":0.37010524522730304,"sd":0.00035545626193963126,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rinalmo","mean":0.45699373559211587,"sd":0.004884092806098253,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"rnabert","mean":0.2783978358579842,"sd":0.0025276350134738963,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.43062462569984683,"sd":0.00023233957648728404,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rnafm","mean":0.43658245827082903,"sd":0.008560152583292163,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.46122343860678233,"sd":0.002260355456973552,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false}]},{"id":"modification","benchmark":"ours_multilabel_classification","name":"Modification Prediction","modality":"RNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.360977051677475,"sd":0.006687271895023917,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"baseline_cnn","mean":0.18592382321208656,"sd":0.001970376727014644,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.3494296759813776,"sd":0.0043555544527369575,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"borzoi","mean":0.19286537152665428,"sd":0.012908350578572295,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.2024492062051651,"sd":0.0033164190876117192,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.20653686742874225,"sd":0.0035921042362724465,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.33419994464534275,"sd":0.0025368340080755736,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"dnabert-3","mean":0.36266578348646683,"sd":0.009068583999101413,"seeds":[1,2,4],"note":0,"rank":7.0,"top_score":false},{"model":"dnabert-4","mean":0.36281768504256273,"sd":0.004022958117416723,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-5","mean":0.35290739180890096,"sd":0.005857631653036996,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-6","mean":0.3628247455175572,"sd":0.00443743970108731,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert2","mean":0.12009909992705897,"sd":0.0023996404293269245,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"enformer","mean":0.13962813521144352,"sd":0.0005879881143934634,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"evo2_1b_base","mean":0.32847731771369576,"sd":0.003924450428334903,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.1571072367698767,"sd":0.0017338595770146354,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.1958990452525478,"sd":0.005128138728024695,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.16698904780371093,"sd":0.0012233351163682298,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.16712347603418387,"sd":0.002652224976562614,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.20973587499351518,"sd":0.0012702774271506124,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.1678577906537971,"sd":0.006625157735281325,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.3653816240137028,"sd":0.008092163595593378,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.3410281278753254,"sd":0.007915648680367928,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"grover","mean":0.23964428882333877,"sd":0.004336003365996767,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"hyena_large_1m","mean":0.33983750900555343,"sd":0.0010676289043590607,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3426390283509964,"sd":0.0048047117361240815,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.21700091780275124,"sd":0.02416224013802646,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.31558616079238117,"sd":0.004392983141532856,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.32406950224410075,"sd":0.0025963113236823487,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.32751556278702687,"sd":0.0058449194095853505,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.34944998261152505,"sd":0.0021283562280037275,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.36728631142972046,"sd":0.0040931549880926605,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.33728642719632423,"sd":0.002455831618405226,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.33215909899266466,"sd":0.0010475048772682636,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3380733162684257,"sd":0.004350328417202526,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3485766689716094,"sd":0.013381945895804631,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.3468615087524637,"sd":0.0025329706673110188,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3330010840594671,"sd":0.0030747145951364333,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.14320192800471293,"sd":0.008769152941241603,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rinalmo","mean":0.3638977949041434,"sd":0.0047227397346480595,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.3037467715381367,"sd":0.006079250734020022,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnaernie","mean":0.33375615219804755,"sd":0.009773502859909187,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"rnafm","mean":0.34594881381568043,"sd":0.012217589749514226,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"splicebert","mean":0.3692164247974451,"sd":0.002390897622148776,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true}]},{"id":"promoter_annotation","benchmark":"ours_multilabel_classification","name":"Promoter Annotation","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9871470019700931,"sd":0.000712399636305596,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"baseline_cnn","mean":0.7293221621694733,"sd":0.007006689826076775,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.673457455551925,"sd":0.025217412883280492,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.816214343623991,"sd":0.0025374171221331956,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6624689689400635,"sd":0.007764380503587459,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7968746864780237,"sd":0.020179387049451655,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.8908865940415106,"sd":0.08714634682683983,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert-3","mean":0.9838171856215161,"sd":0.0029191870621978733,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-4","mean":0.9875185070507453,"sd":0.0010249497818852658,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"dnabert-5","mean":0.9849579245666228,"sd":0.0018379948384649718,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"dnabert-6","mean":0.982605002042307,"sd":0.003948843104441739,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert2","mean":0.6660591024018864,"sd":0.001209471141893989,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"enformer","mean":0.722606360671117,"sd":0.0019098060623762947,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8298461337054748,"sd":0.07495733546687017,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7405377276129889,"sd":0.003671105622047706,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7815971356982209,"sd":0.012841529446945015,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7958279002919086,"sd":0.03185689873462856,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7606745465496607,"sd":0.012336874091531261,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.8427897680166866,"sd":0.022399091902064037,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.8110934983551954,"sd":0.03769229417576553,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9524556154756011,"sd":0.0029147821654875146,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"phylogpn","mean":0.5762913781667958,"sd":0.004878183334788081,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"grover","mean":0.8485140005822203,"sd":0.005571817317963897,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6620189343209468,"sd":0.011668653653765224,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.724261632007833,"sd":0.034843338015399974,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6544196793483698,"sd":0.0276521087779686,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9247823113748733,"sd":0.00944774993105401,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9591073895786014,"sd":0.00044496933864594004,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9175044070129704,"sd":0.004767970013727319,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9227532075074143,"sd":0.003016245927759819,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9482785133859054,"sd":0.002333181355422956,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8818510466934832,"sd":0.0025810027143497085,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.87631273469997,"sd":0.007493717112018663,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9144686758700504,"sd":0.0007692247256467869,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9299647902172664,"sd":0.0013561991889610154,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9354132499058841,"sd":0.0015003523383982315,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.885497372480916,"sd":0.0162744092991175,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"orthrus-4track","mean":0.5536359670849863,"sd":0.00683330688579029,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rinalmo","mean":0.9801216833363203,"sd":0.0006749119268312455,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.6548228794615283,"sd":0.014404318585402762,"seeds":[1,4,5],"note":null,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.9863119109521703,"sd":0.00149920198708267,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"rnafm","mean":0.9744276684923516,"sd":0.004316881675827493,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"splicebert","mean":0.9751533072118759,"sd":0.0012490873860483837,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false}]},{"id":"regulation_multilabel","benchmark":"ours_multilabel_classification","name":"Regulation Element Prediction","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8016509788445666,"sd":0.0012583006035676739,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6907153326076939,"sd":0.0014168698873802694,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.732071754994668,"sd":0.007556254843920335,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"borzoi","mean":0.7909165127171716,"sd":0.0030581346427426747,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7721014097771891,"sd":0.0013590898847622576,"seeds":[1,3,6],"note":11,"rank":5.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7650124429448948,"sd":0.002629071744906618,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.7394018035777706,"sd":0.0005625376640172226,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-3","mean":0.7026012234266205,"sd":0.0015344261914643769,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-4","mean":0.7059877080956112,"sd":0.0011062714864458845,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-5","mean":0.704834249238997,"sd":0.0022371215718221763,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.7081596000748653,"sd":0.0025550603550431477,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.7302503901446236,"sd":0.002408774628652732,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"enformer","mean":0.7902765352061817,"sd":0.0036204032299194545,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7586006289442153,"sd":0.003255919637088102,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7313436917657521,"sd":0.0013574163335168016,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7530099744895836,"sd":0.0018065169864927026,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7446906107981439,"sd":0.0027242375358243467,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7430067145578345,"sd":0.004238746190824031,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.754809434629617,"sd":0.0021271480834481977,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7500568701330832,"sd":0.0016993364666950874,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7708935106787489,"sd":0.0036554378749324057,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"phylogpn","mean":0.7154042236004322,"sd":0.0010612547871100496,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"grover","mean":0.7459305243323052,"sd":0.0012011505782805586,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7476809342797851,"sd":0.0034758572571903655,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7422917766226198,"sd":0.003788348959878984,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6581342081645369,"sd":0.05444662808779188,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7632260163588576,"sd":0.0008023047829114443,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7705608442805391,"sd":0.0020993402128842865,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7596481047706616,"sd":0.004496577643272052,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7556835616133092,"sd":0.0018859855392271258,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7716411326122571,"sd":0.000671107463247226,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7413095307502914,"sd":0.0009231002350164,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7404974356034268,"sd":0.0005635606494941054,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7641744591874562,"sd":0.001177327834357672,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7697862393861388,"sd":0.0026457241802865837,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7762788956586315,"sd":0.0005557183841315518,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7535809397336947,"sd":0.002048675346994148,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.727159560924662,"sd":0.0012958177027917546,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.766780553943482,"sd":0.0009373948511536637,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"rnabert","mean":0.5219107301811703,"sd":0.0006490730310342634,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7555860782486808,"sd":0.003226440363670111,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"rnafm","mean":0.7581276990220623,"sd":0.003314096077881387,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"splicebert","mean":0.7668915506735757,"sd":0.0012421609303612975,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false}]},{"id":"splice_sites_all","benchmark":"ours_multilabel_classification","name":"Splice Sites Prediction","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9937522747080093,"sd":0.0003042283904866725,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.9639757390621674,"sd":0.0007938883674722137,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.9629590032195715,"sd":0.021348790448105174,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"borzoi","mean":0.9826588224449403,"sd":0.000754126443237543,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8817869294191611,"sd":0.018079770367388275,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.9180522487478251,"sd":0.006200461231699273,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.9864610986919037,"sd":0.0018675853458127798,"seeds":[1,2,4],"note":0,"rank":21.0,"top_score":false},{"model":"dnabert-3","mean":0.9906320397270224,"sd":0.00015046333706842828,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-4","mean":0.9908030677374831,"sd":0.0004550764287920665,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"dnabert-5","mean":0.9894006188734771,"sd":0.000664750840556487,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-6","mean":0.9907534215559776,"sd":0.0007588250691072302,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert2","mean":0.9165443932252894,"sd":0.005837437605685356,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.9757449910445677,"sd":0.0004792474240470051,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"evo2_1b_base","mean":0.990455238340507,"sd":0.0005046110272128218,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.9472971049590388,"sd":0.007886398048897061,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.9591952233896108,"sd":0.0008572136121388128,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.9658817490519211,"sd":0.0008740969485495636,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.962584086999514,"sd":0.0008904270951350604,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9708186022231291,"sd":0.003299474038240548,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.9686088339306526,"sd":0.0018878159311179748,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9930727857624223,"sd":0.00045054594836925883,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.9573672245298946,"sd":0.0006851252616775227,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"grover","mean":0.9676822398604226,"sd":0.002665471757185723,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_large_1m","mean":0.9728626261736592,"sd":0.003850799827928907,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.9716726274317616,"sd":0.001784690925224903,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.9399711702220968,"sd":0.004160270714481458,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9900289995766225,"sd":0.00022149215969295293,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9901256740418071,"sd":0.0015312056052965103,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9894180089427915,"sd":0.0005934908336063705,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9893081039820851,"sd":0.00031122445454375515,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.992898115398184,"sd":8.975135313965303e-05,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.9849790795356728,"sd":0.0005548502839257115,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.9808536648315082,"sd":0.0005227305138901385,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9904302151030914,"sd":0.0010601337516023563,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.992327644632133,"sd":0.00019018596653531597,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9934592575454309,"sd":0.000401069060835262,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.9893045622463994,"sd":0.0005088773222960291,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":0.7786633475273533,"sd":0.002909593852056896,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rinalmo","mean":0.9932264671540818,"sd":0.00025923443770683487,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"rnabert","mean":0.9735192446821326,"sd":0.0005428972896758866,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnaernie","mean":0.9900844957800936,"sd":0.00020035607422166186,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"rnafm","mean":0.9874753533674344,"sd":0.0005287274643941905,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.9927127910514782,"sd":0.00036631723158644643,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_gm12878","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.675328303555444,"sd":0.019929640554609662,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.6052464843327422,"sd":0.029719228336524663,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.6107065371885447,"sd":0.016493280299653766,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"borzoi","mean":0.7470274074093338,"sd":0.005367454632516932,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6785345628597798,"sd":0.02351403130887446,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6731206476003291,"sd":0.018232452041112585,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6377814703525112,"sd":0.016190765592513572,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"dnabert-3","mean":0.5056953790893369,"sd":0.05579647954613109,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-4","mean":0.5620251495598864,"sd":0.024593240947658915,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.6024293171356635,"sd":0.019916448100295206,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-6","mean":0.5596076025120752,"sd":0.037730526045663154,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert2","mean":0.642441767302403,"sd":0.009566312954305536,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"enformer","mean":0.7181335606392761,"sd":0.01680362041772825,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6621976543218644,"sd":0.043005505107343134,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6720961547551243,"sd":0.03015515939882238,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6931273214954548,"sd":0.016253702087382825,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6111234248031442,"sd":0.031205202888352355,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5984230802327265,"sd":0.06002160445192675,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6993269478249798,"sd":0.03452777119064079,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6333840336369725,"sd":0.0456263060027448,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6354155075863266,"sd":0.03132820258889813,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"phylogpn","mean":0.6409886153679456,"sd":0.029361464995443606,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"grover","mean":0.6939776140600963,"sd":0.007156073887350856,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6496962427030779,"sd":0.005721689730423687,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6435504102414545,"sd":0.01019228387586712,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5724631351027977,"sd":0.02268632490751052,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6081822922390108,"sd":0.050116697444446676,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6646174490378195,"sd":0.01064845925233372,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6336634468247215,"sd":0.017118465855079226,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.67179888340238,"sd":0.01675101891416809,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7222894466738762,"sd":0.01883071378274015,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6615482999129715,"sd":0.039034058097266774,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6900896133785681,"sd":0.022497240427423324,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6421219033908322,"sd":0.008525628550783138,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6709018875200713,"sd":0.015870688296976992,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6815580248444232,"sd":0.05288630861112588,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.646521386234924,"sd":0.02040965718551252,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.6021340622474042,"sd":0.00885700997099873,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rinalmo","mean":0.4772403689628979,"sd":0.00025385925717968114,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnabert","mean":0.4960031914621214,"sd":0.021599935777402136,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.6911678515333747,"sd":0.036273001650417976,"seeds":[1,2,4],"note":null,"rank":7.0,"top_score":false},{"model":"rnafm","mean":0.680025074400347,"sd":0.010487834538048118,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"splicebert","mean":0.6769967338885661,"sd":0.002760891992767321,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_hela_s3","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5809383160410339,"sd":0.013602761701820731,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"baseline_cnn","mean":0.6031710180904845,"sd":0.00889778843555526,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"baseline_lstm","mean":0.5136698729358592,"sd":0.008495618593039723,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.6467657269145737,"sd":0.018799329469606718,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6480417287358053,"sd":0.003251588822086107,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6205726268989403,"sd":0.03316214896864654,"seeds":[1,4,5],"note":6,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.48945227675396463,"sd":0.034804468176367405,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.5446405584540753,"sd":0.04065984407430754,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.6134255428804678,"sd":0.009450704241923492,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"dnabert-5","mean":0.6127161338112694,"sd":0.026211192193519924,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert-6","mean":0.5494382027721323,"sd":0.06561911872307744,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert2","mean":0.6170798222120041,"sd":0.03503382362868238,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"enformer","mean":0.5731301755475748,"sd":0.016414201562058223,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6535663335215953,"sd":0.06193534077456404,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6663300868179259,"sd":0.03549570935708846,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6290189980491031,"sd":0.010195393645103907,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5310010559607145,"sd":0.028167927003842824,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5756140493371622,"sd":0.03076036898345458,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6118999630198453,"sd":0.019307597814654718,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4822409432412948,"sd":0.0026305500854426666,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6088383812006926,"sd":0.02317834783880805,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"phylogpn","mean":0.6010476791965864,"sd":0.012214885518264055,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"grover","mean":0.6761734452717675,"sd":0.007691289436688497,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5813477888296463,"sd":0.016087083249220303,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6303670342070374,"sd":0.0065856835678994455,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.4790439866147498,"sd":0.007028275559298221,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6163408545267248,"sd":0.01938737251048351,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5527531976241203,"sd":0.024263422599865293,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6046700626131207,"sd":0.027421182564874833,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6666779722937327,"sd":0.02058108907302371,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6572218525435776,"sd":0.027012311587456764,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7100885939196926,"sd":0.029783891381366633,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6749244309090761,"sd":0.014673598285738497,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5447518061518106,"sd":0.01627475466686738,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5840554526130666,"sd":0.025452309902980412,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6046280183868922,"sd":0.033963423316637056,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5850091903216925,"sd":0.026482072119316292,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"orthrus-4track","mean":0.5267635440473409,"sd":0.02299241166674061,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rinalmo","mean":0.4752722164074249,"sd":0.008183161104090104,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnabert","mean":0.5152151873680297,"sd":0.027536589060748476,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rnaernie","mean":0.6975810136340299,"sd":0.006226815212949183,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"rnafm","mean":0.6754893259808378,"sd":0.02141947565935146,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"splicebert","mean":0.6615934703587585,"sd":0.012904739008501722,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_huvec","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6413032040726242,"sd":0.00954286762008842,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.6313061357202466,"sd":0.014743426445574172,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"baseline_lstm","mean":0.5439492958635586,"sd":0.004305469150675116,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.6511177043737933,"sd":0.02702971122303881,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6487982356839279,"sd":0.01680205890237351,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6698670730523888,"sd":0.03451357033678746,"seeds":[1,3,4],"note":6,"rank":4.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5788869424410021,"sd":0.02816265306649979,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-3","mean":0.49402482937779574,"sd":0.019595127983196115,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-4","mean":0.5593391350183394,"sd":0.007595985858326088,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-5","mean":0.5579712613599783,"sd":0.02995757606628563,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-6","mean":0.5052841040592543,"sd":0.05837320445917917,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert2","mean":0.61440728145091,"sd":0.03512401464998915,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"enformer","mean":0.678708344705382,"sd":0.011559229310484951,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"evo2_1b_base","mean":0.6270437233329386,"sd":0.057289980225707145,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6292917591230396,"sd":0.03187467180443253,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.665457166944931,"sd":0.0048079921546100605,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5843220818614048,"sd":0.038183791409263486,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5680721583043031,"sd":0.020676429306516815,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.58767408582043,"sd":0.009205176177493736,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6346802763109834,"sd":0.031700126485732755,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6555336225579214,"sd":0.052588948359967164,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"phylogpn","mean":0.6176094177973858,"sd":0.0022018619023690083,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"grover","mean":0.636675844928373,"sd":0.02391146748015363,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6347142805850681,"sd":0.027274974922083906,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6431717484454591,"sd":0.056221163408849265,"seeds":[1,2,3],"note":12,"rank":13.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5487499024390813,"sd":0.03725629636557086,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6240001406861587,"sd":0.010015292385955728,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6411552784199388,"sd":0.02278613507629561,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6203358629816684,"sd":0.02745771488358368,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6730848454066773,"sd":0.005437943396603107,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6638512025442018,"sd":0.018934075808975076,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6616599545608709,"sd":0.018260901696466287,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6755781259651977,"sd":0.01134392020569479,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5932306352675082,"sd":0.01702466819172435,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6309095789533544,"sd":0.016048464919191506,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6692078562229379,"sd":0.029983526886255236,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6130448368650804,"sd":0.032669220425239495,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"orthrus-4track","mean":0.5590041803469128,"sd":0.020255381564149782,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rinalmo","mean":0.47327257403299544,"sd":0.005882885540876845,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnabert","mean":0.4769141018693788,"sd":0.0047355417464495135,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.6558247770808138,"sd":0.005414790981576599,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnafm","mean":0.627988662135411,"sd":0.011456945435305156,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"splicebert","mean":0.6240311868555909,"sd":0.026033776018331393,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_imr90","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5527448302094978,"sd":0.04600210170215114,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"baseline_cnn","mean":0.5660257544637533,"sd":0.0035936453171778516,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"baseline_lstm","mean":0.5138461142542765,"sd":0.01299861836437968,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.6574921144003817,"sd":0.027726106038396252,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5849829925020498,"sd":0.01965554751632172,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5475884982440435,"sd":0.03101234325107593,"seeds":[1,4,5],"note":6,"rank":30.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.4504623293325874,"sd":0.0027755822213789603,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.5689250433043489,"sd":0.014273508971970095,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"dnabert-4","mean":0.5542390810114742,"sd":0.01522369369327912,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-5","mean":0.520172643010198,"sd":0.04085509499693973,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-6","mean":0.5389697966272575,"sd":0.015704326247909343,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert2","mean":0.46933565812413114,"sd":0.010808127416970138,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"enformer","mean":0.6278654343012251,"sd":0.03829134630134017,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5452267570860018,"sd":0.06382358861894048,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5911539969356443,"sd":0.02035445323410254,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6305869026887838,"sd":0.021045763971004765,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5739695964708067,"sd":0.03334249162813509,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5925827644902478,"sd":0.05549678148083024,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6680739442261303,"sd":0.06110536494992449,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5131648308223657,"sd":0.04872141200402399,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6178507767447636,"sd":0.020095090461332753,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"phylogpn","mean":0.5894841889958865,"sd":0.03148910547477186,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"grover","mean":0.55450073507053,"sd":0.022657333889596655,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"hyena_large_1m","mean":0.545493908824989,"sd":0.03844171295187838,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5225587101078013,"sd":0.023831713419676204,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5131696767032711,"sd":0.02738178285063294,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5588058806375918,"sd":0.013125756873971035,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5958922666343223,"sd":0.09874851451664197,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5725422530555979,"sd":0.017302842026385788,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6903359513166618,"sd":0.07332558255242173,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7450754472971537,"sd":0.0259643485493418,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-1000g","mean":0.6517771825993615,"sd":0.02609807545799095,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6427321939360856,"sd":0.03251097897892402,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5857340002542273,"sd":0.02390544073855646,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6168218710223086,"sd":0.03147625429216474,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6235546378012667,"sd":0.049315499549332585,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5607489125061155,"sd":0.025257242162530755,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"orthrus-4track","mean":0.47116140060215866,"sd":0.010286270187860031,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rinalmo","mean":0.471190948589788,"sd":0.00873430964264755,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnabert","mean":0.5155765868355721,"sd":0.002610643285409505,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rnaernie","mean":0.6246264367152993,"sd":0.03976174817103748,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnafm","mean":0.6064323249027731,"sd":0.023224935820115363,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"splicebert","mean":0.676777955837729,"sd":0.01632271255451209,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_k562","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5983644473371226,"sd":0.037798202099307156,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"baseline_cnn","mean":0.554374339976322,"sd":0.018312030763620216,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.5449591394358058,"sd":0.029408795190256902,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.6260911061825617,"sd":0.016562331615864366,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6234868663343393,"sd":0.01385602084475482,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6238910812368327,"sd":0.016988585006276178,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5307567749588632,"sd":0.019115829693917816,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.46973585531024353,"sd":0.011610361765043731,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"dnabert-4","mean":0.5421651430941804,"sd":0.015662553531273566,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-5","mean":0.5387798491270973,"sd":0.014492944084399358,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-6","mean":0.5425050596004457,"sd":0.00864628750181795,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.5743867855263312,"sd":0.008027318706165663,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.6194803092207448,"sd":0.014810474482879239,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5902559957937666,"sd":0.028457037148506784,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6140538336399034,"sd":0.024397402313275755,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6068452709678475,"sd":0.00674132833045986,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5595157755645358,"sd":0.018363731050978233,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5657938401089176,"sd":0.046993656661710966,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6181436073879681,"sd":0.017833936256066756,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5475608420816616,"sd":0.04730292992509591,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.627420968115587,"sd":0.027279552464293227,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.6280617378033818,"sd":0.025744693387910286,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"grover","mean":0.6073797644114762,"sd":0.013551823584460273,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5648902037589163,"sd":0.0751145370077871,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5954258205222711,"sd":0.04400385587014288,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5285262401942343,"sd":0.05171467167588243,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5992486946725918,"sd":0.032828491396512165,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.566248865101841,"sd":0.03472638899148512,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6054459037774553,"sd":0.025960160680833344,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6045213865257548,"sd":0.042003055667281,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6318808946218551,"sd":0.010400669464531451,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5854129470082493,"sd":0.03998091469229502,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6100991771515786,"sd":0.044346902666181715,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5611784721185111,"sd":0.011348305172208136,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.60562779650355,"sd":0.009874620042539922,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6347520083765196,"sd":0.024840989177879687,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5954935234045127,"sd":0.02124534529223119,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"orthrus-4track","mean":0.564934284663024,"sd":0.024272569068667278,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.4771035800791007,"sd":0.004356413240040706,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnabert","mean":0.48862236855284147,"sd":0.009337763266899774,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.5848639107195811,"sd":0.013390595629360342,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"rnafm","mean":0.5974605186819638,"sd":0.02152392592883931,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"splicebert","mean":0.5894376419094595,"sd":0.007523174510514973,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_nhek","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5768638181237348,"sd":0.025175188900152058,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"baseline_cnn","mean":0.6117202483193939,"sd":0.02395610256137416,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_lstm","mean":0.5020913948181355,"sd":0.022912542424851023,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.728924435474175,"sd":0.04844401672557155,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5774018838346489,"sd":0.01635935819889713,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5380686732074079,"sd":0.01962391454835831,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5011957438350161,"sd":0.03136236900122872,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-3","mean":0.5372334075778195,"sd":0.04854867976216488,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-4","mean":0.6004901403749089,"sd":0.020834128918178627,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert-5","mean":0.47416603443972843,"sd":0.005414387456631951,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-6","mean":0.5928827128521682,"sd":0.012544006032427698,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert2","mean":0.49041811642104327,"sd":0.010813027238463842,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"enformer","mean":0.638567366357745,"sd":0.017165514866254512,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"evo2_1b_base","mean":0.560196462098678,"sd":0.10267079342879547,"seeds":[1,2,4],"note":0,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5634025635198139,"sd":0.01601672523672616,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6891507121667825,"sd":0.03792250038817373,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5648730457731812,"sd":0.05970771487663268,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5651048932251258,"sd":0.03987736975338131,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5810301379317118,"sd":0.0006164679650658572,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.47925142529345166,"sd":0.0074808668891257835,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6019599548437367,"sd":0.02947546268427728,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"phylogpn","mean":0.5292310080153523,"sd":0.02206479019929393,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"grover","mean":0.6879136418298852,"sd":0.05024397879102379,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5273716317558189,"sd":0.02212596915729408,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6117717171623048,"sd":0.044993434645845605,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.46991362168254286,"sd":0.013477421585762744,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5697373622640632,"sd":0.01062729290057787,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6227250413410992,"sd":0.07449141451991244,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5793759201228847,"sd":0.039227798528869456,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6763846180162082,"sd":0.04047884416742111,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6731233948590022,"sd":0.06313427620382543,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6594906140653707,"sd":0.05100936680510736,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6827152691452112,"sd":0.050988786204454746,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5343333738204794,"sd":0.012663482863511516,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5481180825399867,"sd":0.006473126132308665,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5612719504771502,"sd":0.034869828662694884,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5555861760060355,"sd":0.047449293365126176,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"orthrus-4track","mean":0.5071809293016248,"sd":0.022870234783443077,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.483921929668365,"sd":0.00223884807656763,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"rnabert","mean":0.5145348413884248,"sd":0.050632419618371706,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rnaernie","mean":0.4838959283780957,"sd":0.005056061035273378,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnafm","mean":0.641897703771206,"sd":0.028781622028443175,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"splicebert","mean":0.688334843211433,"sd":0.018827960643605715,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false}]},{"id":"enhancer_promoter_interaction_gasperini_k562","benchmark":"ours_paired_classification","name":"Enhancer Promoter Interaction Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"paired_3kb_2kb_nonoverlap_component_holdout_v1","results":[{"model":"alphagenome","mean":0.13452407009105236,"sd":0.05975180252203619,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"baseline_cnn","mean":0.10598033231015051,"sd":0.04412860011492427,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"baseline_lstm","mean":0.08364600505399744,"sd":0.001596436910544194,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"borzoi","mean":0.1537557421515254,"sd":0.03563036281784134,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.07520596340736739,"sd":0.0010480510299794151,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.07504680317145546,"sd":0.012134971476060623,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.07678478496525595,"sd":0.009566720030082883,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.09462536183475895,"sd":0.020891979622166264,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-4","mean":0.10523400014031757,"sd":0.024001641630055196,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-5","mean":0.11776391612222835,"sd":0.012415682358688122,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert-6","mean":0.11097062924896423,"sd":0.0295635238292172,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert2","mean":0.09231915497127238,"sd":0.010798827219643075,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.19851793778157278,"sd":0.030752241182863786,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"evo2_1b_base","mean":0.11952216030407586,"sd":0.038088017589633434,"seeds":[1,2,3],"note":13,"rank":6.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.08965993730811021,"sd":0.011473201195629697,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.09072870587403364,"sd":0.0060099410942149175,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.0930729706366752,"sd":0.014898452488596215,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.10857902195517119,"sd":0.040651317723339936,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.11688772958265559,"sd":0.01716127786557885,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.12517639333614253,"sd":0.03714895835717367,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.10845089636501937,"sd":0.00408610390784483,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"phylogpn","mean":0.11609292561668186,"sd":0.031075491130195098,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.09314243206278595,"sd":0.026045917890481354,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.0775447621135638,"sd":0.01380292052385182,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.07102837407215389,"sd":0.002670402018432164,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.09490493547622611,"sd":0.018093801655501222,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.1074570066952629,"sd":0.027976435161960283,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.09781955163954206,"sd":0.024226455450412195,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.11213461090540246,"sd":0.029901128114415164,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.09849172798432955,"sd":0.0054296608832109195,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.13067390553226918,"sd":0.03320422703838065,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.0893131736982672,"sd":0.005817172118537821,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.09209287300609788,"sd":0.004008098130909434,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.08136074831875463,"sd":0.0066492219517409416,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.11570251016563657,"sd":0.016641158781007703,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.09300068581519227,"sd":0.003199709016728377,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.09655250435307688,"sd":0.0017937914428818,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"orthrus-4track","mean":0.08141746852940657,"sd":0.002744858273916839,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.07307819243290284,"sd":0.001637691804205934,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnabert","mean":0.07755189439789166,"sd":0.006640588711120378,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rnaernie","mean":0.08788613948129756,"sd":0.002385972813856824,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rnafm","mean":0.09097318079236542,"sd":0.009875273048468155,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"splicebert","mean":0.0911369253522711,"sd":0.018995386073461074,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false}]},{"id":"beta_lactamase_complete","benchmark":"ours_regression","name":"Beta-Lactamase Activity Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.4678137251478634,"sd":0.016683635193506935,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"baseline_cnn","mean":0.5741604934644955,"sd":0.02173944326836539,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"baseline_lstm","mean":0.15944367990544905,"sd":0.021549843149473207,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.6321904323872127,"sd":0.009778859590516041,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.028575604407110287,"sd":0.012571843856236435,"seeds":[1,2,3],"note":14,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":-0.015300670792392883,"sd":0.012986716224311582,"seeds":[1,4,5],"note":15,"rank":43.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.02128341016067697,"sd":0.0725650341316024,"seeds":[1,3,5],"note":15,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.2794576665707854,"sd":0.030245844677716786,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.30767392280072203,"sd":0.006256520093961062,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-5","mean":0.2801844009331054,"sd":0.01062562155908903,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.2817590551083299,"sd":0.010502396499581998,"seeds":[2,3,4],"note":0,"rank":30.0,"top_score":false},{"model":"dnabert2","mean":0.3345986436813244,"sd":0.020002880663817054,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"enformer","mean":0.5336676009918345,"sd":0.028007306908295657,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"evo2_1b_base","mean":0.0036947819156317313,"sd":0.03912910459140081,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.41740319217848904,"sd":0.0577369270722931,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5133965519852346,"sd":0.01464241188798054,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3457524877117493,"sd":0.015006407471463652,"seeds":[1,2,4],"note":0,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.29412655350407974,"sd":0.04157275868368473,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.38672618181182844,"sd":0.03291479841291443,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3780915840711702,"sd":0.09370830427098926,"seeds":[2,3,4],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7860131254990118,"sd":0.04093141174703661,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.6839725257414235,"sd":0.005301118353096519,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.47275439985746903,"sd":0.008655713152003934,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.042457114047862475,"sd":0.06140950168019351,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.09464315709939082,"sd":0.05037301466643332,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.08766252437458198,"sd":0.033215215584936134,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6906107101104553,"sd":0.026749870830622312,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7221610349544823,"sd":0.07740804253930682,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5906819643582008,"sd":0.12464129372372434,"seeds":[1,3,4],"note":0,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6986936562780816,"sd":0.004298373781055066,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7849676551195701,"sd":0.03203896091381672,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5900402739641755,"sd":0.025101469594779255,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6130399989529408,"sd":0.004723869301132776,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6912321277197111,"sd":0.009235628081195785,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.764288056711739,"sd":0.011667614204683421,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7253666002366943,"sd":0.016375326455892493,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6054215448555418,"sd":0.007964004164960643,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"orthrus-4track","mean":0.1948560240737637,"sd":0.0689628244431036,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.6472128432457535,"sd":0.030605862775236747,"seeds":[1,3,4],"note":0,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.08602644466970015,"sd":0.06472295987225818,"seeds":[1,2,3],"note":16,"rank":38.0,"top_score":false},{"model":"rnaernie","mean":0.4910293213332263,"sd":0.046335093099746844,"seeds":[1,3,4],"note":null,"rank":19.0,"top_score":false},{"model":"rnafm","mean":0.2308767345835554,"sd":0.04822404754415637,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.44532221399562677,"sd":0.005479832439636087,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false}]},{"id":"beta_lactamase_unique","benchmark":"ours_regression","name":"Beta-Lactamase Activity Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.33822357117403595,"sd":0.03960875173162129,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"baseline_cnn","mean":0.5060610697480262,"sd":0.018088013309114984,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_lstm","mean":0.11360489817342825,"sd":0.01167352985865761,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"borzoi","mean":0.6159088674409076,"sd":0.0184478256233833,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.01602238468339103,"sd":0.035003590699173394,"seeds":[1,2,3],"note":17,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":-0.0035004850253409097,"sd":0.0415306704272969,"seeds":[1,4,5],"note":15,"rank":42.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.02586098744336303,"sd":0.0026892661444568905,"seeds":[1,4,5],"note":18,"rank":38.0,"top_score":false},{"model":"dnabert-3","mean":0.18455290865340293,"sd":0.03591623524155839,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"dnabert-4","mean":0.16531056200103372,"sd":0.021858616405157225,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"dnabert-5","mean":0.14217164987683886,"sd":0.017293729616620304,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.14499111987236482,"sd":0.03401157861170632,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert2","mean":0.15876829259276035,"sd":0.12802067416729096,"seeds":[1,3,4],"note":0,"rank":29.0,"top_score":false},{"model":"enformer","mean":0.28194924317187725,"sd":0.013994508871334408,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"evo2_1b_base","mean":0.01903833388893039,"sd":0.0051791441000957165,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.3234277035494053,"sd":0.0954040944051055,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.24518377535294963,"sd":0.021754137875438742,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.13492552995009421,"sd":0.07048912472748355,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.16200657848502875,"sd":0.0770089513734725,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.31704325879726797,"sd":0.040915190755282584,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.22031826676800473,"sd":0.1213128874832408,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6769809046925029,"sd":0.04010879196042051,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.5990425600445667,"sd":0.0038401828599575892,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"grover","mean":0.39005209269979874,"sd":0.03210905079842785,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":-0.024095688274965735,"sd":0.034081013873606354,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.05932470321731812,"sd":0.021851384305002174,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.11634690292657512,"sd":0.011693314826556502,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5808624704971469,"sd":0.0105039697873596,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6137260226246778,"sd":0.06852051756644331,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.48799461990468024,"sd":0.0057333980004846625,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5776838252385155,"sd":0.0032932249072381366,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6528774998696071,"sd":0.05223353381766104,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.4338312603369034,"sd":0.0345931599045018,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5073320889742311,"sd":0.00967323419359772,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5402413112959229,"sd":0.01904691668196746,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5826086807382288,"sd":0.051600539226885435,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5954439744270019,"sd":0.06840880838092978,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.437555103908805,"sd":0.035331760723653315,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"orthrus-4track","mean":0.10221926729962784,"sd":0.04707420163400249,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.5816866019018198,"sd":0.008933594939004499,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.006641822575385359,"sd":0.08418158766974307,"seeds":[1,3,4],"note":19,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.4313569855656573,"sd":0.019159901472978667,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"rnafm","mean":0.10055363110454851,"sd":0.05525913139036777,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"splicebert","mean":0.2590222199432931,"sd":0.024054414529005554,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false}]},{"id":"bulk_rna_expression_1024","benchmark":"ours_regression","name":"Gene Expression Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8365649664079731,"sd":0.0023697901045889873,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.707732600794415,"sd":0.0024339432897907912,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.7163252929749567,"sd":0.0015809437588975616,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"borzoi","mean":0.7887914004094589,"sd":0.004529363549399618,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7336604827294898,"sd":0.0016035547235427237,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7323334146784849,"sd":0.00223627547862708,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.7052865509359663,"sd":0.004368592988973825,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-3","mean":0.7066826595082929,"sd":0.0008664982405637102,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.7041155184740349,"sd":0.0009063046014639394,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.7041078454157322,"sd":0.00595808024642362,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-6","mean":0.7012571866249088,"sd":0.002438170655680208,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert2","mean":0.739686155380794,"sd":0.004968014221051481,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"enformer","mean":0.7754563811636939,"sd":0.002438802226354179,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7298263746938137,"sd":0.005250870476061586,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7202264348113655,"sd":0.0010888142883520699,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7488889567440319,"sd":0.001746049245330425,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7080026070754832,"sd":0.002944561122850661,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7101148734811781,"sd":0.0033806899583762816,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7558398393402528,"sd":0.005993146900166727,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7513620027802002,"sd":0.004980019576562016,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7857445161057312,"sd":0.002195252579171815,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.7407701530376717,"sd":0.0027177014157400035,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"grover","mean":0.7400660745894153,"sd":0.0037363206337166443,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7335490899024927,"sd":0.006301932556868208,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7277874656620291,"sd":0.0014053145248820234,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6850573201827822,"sd":0.0030068106071118555,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7447647803606503,"sd":0.004138790503363926,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7660478319328519,"sd":0.0020745395046646914,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7491659921125908,"sd":0.0024176941823237487,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.754118868260793,"sd":0.005219893511284676,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7708632503967378,"sd":0.0024884688900518705,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7259613963679604,"sd":0.003967481449485549,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7133495776490288,"sd":0.0015762335823001107,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7508840630977606,"sd":0.005047348020942298,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7554121262687463,"sd":0.002252242114491621,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.77223813164015,"sd":0.002889258323193831,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7351685772874863,"sd":0.005478190362506142,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.7131466929048242,"sd":0.0026508673029010913,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rinalmo","mean":0.7518900724143128,"sd":0.005471672615593505,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"rnabert","mean":0.5677932028084116,"sd":0.002001731757077645,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7298196189166459,"sd":0.0070637117302172515,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"rnafm","mean":0.7411590766299111,"sd":0.001991131236036247,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"splicebert","mean":0.764785532730916,"sd":0.002542968629250704,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false}]},{"id":"bulk_rna_expression_128","benchmark":"ours_regression","name":"Gene Expression Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.7499046911436532,"sd":0.00232674597007269,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6738839300876439,"sd":0.005136375790649047,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.6869604549513083,"sd":0.00266582934156726,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"borzoi","mean":0.7242397024624012,"sd":0.008609749186076649,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6852311362810007,"sd":0.0020721998291227976,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6798403786871476,"sd":0.004232662925234606,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6712804319641851,"sd":0.005169179988381369,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.6995003999095503,"sd":0.003699711306951853,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-4","mean":0.7127448498778539,"sd":0.001953884981773839,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-5","mean":0.7137944407127043,"sd":0.0011470027815704321,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"dnabert-6","mean":0.7045238321825069,"sd":0.0038097442540483474,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert2","mean":0.6692372045570569,"sd":0.008207359627363093,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"enformer","mean":0.7113776725834913,"sd":0.0026237671226445056,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6795219732712984,"sd":0.0025521064151693943,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6703100712547204,"sd":0.0010269521312179584,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6952888112903376,"sd":0.0030123421837907442,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.662038250475612,"sd":0.004121792538416456,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6605424002414768,"sd":0.004341362394619464,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7094020802299799,"sd":0.0031006444286923148,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.689074337059122,"sd":0.003030028657483906,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7064931337114045,"sd":0.005284784745907208,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"phylogpn","mean":0.70729446870843,"sd":0.003229327935952717,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"grover","mean":0.6903658552650785,"sd":0.006428358025356032,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6898884923159824,"sd":0.0035843975648008554,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6863379740834502,"sd":0.005599833023419313,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6307012662815934,"sd":0.012129547134135615,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7062926820799261,"sd":0.0023854182938580454,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7145100953479431,"sd":0.0030325628061912626,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6966393973841507,"sd":0.001816288929635441,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6910809867418553,"sd":0.005169445641333554,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7040212702749251,"sd":0.0011919520411705679,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6678216374584826,"sd":0.006079609165453307,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6668573022230597,"sd":0.002331157335228005,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6804309519914118,"sd":0.00840939428426777,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6988051689792333,"sd":0.0032988405537208764,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7040789405647101,"sd":0.0037341180429660325,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6725944490056626,"sd":0.006086296553555038,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"orthrus-4track","mean":0.6846194454491199,"sd":0.0013652919733081392,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.7079972000467348,"sd":0.00952288841469917,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.6278916565187307,"sd":0.0018770040974392837,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.6854164837374034,"sd":0.001899112956670253,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"rnafm","mean":0.6886865184683667,"sd":0.0049142391771065394,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"splicebert","mean":0.710747996515584,"sd":0.0013003754482312504,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false}]},{"id":"bulk_rna_expression_2048","benchmark":"ours_regression","name":"Gene Expression Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8517604067737605,"sd":0.0005037580356004206,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.7083394837777542,"sd":0.010363328398270488,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.6966541384948579,"sd":0.010325964058763442,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.8107812585895524,"sd":0.0019234663785631378,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7376854565431081,"sd":0.000856999319265444,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7354177580742116,"sd":0.002625226235010977,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.702564130882301,"sd":0.004274848612200679,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.5240173510632836,"sd":0.007330652126495891,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-4","mean":0.5368341110450955,"sd":0.0027805220384594063,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-5","mean":0.5403916771998317,"sd":0.0051131689913882505,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.5384908210575808,"sd":0.0057725186289584365,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert2","mean":0.7480631674514541,"sd":0.002697621706508288,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"enformer","mean":0.7975199391548886,"sd":0.0040966901432515305,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7324951076267172,"sd":0.010794945280091561,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7310530840831253,"sd":0.0016183583415138357,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7538388556553711,"sd":0.0023711058881966313,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7175903862419744,"sd":0.0028449288007295426,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7166821648460587,"sd":0.0044695013289181304,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.771395137036658,"sd":0.008057876049455437,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7536214032271813,"sd":0.001867511296594792,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7827720974180037,"sd":0.00429205839770604,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.7403347615727931,"sd":0.00419384317887942,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"grover","mean":0.7430704680730797,"sd":0.004117429531051917,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7185879343877469,"sd":0.018218112313742155,"seeds":[1,3,4],"note":10,"rank":29.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7224567866421194,"sd":0.008508680559076321,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6866394193978822,"sd":0.00431106932485928,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7570953470437465,"sd":0.007301478169916484,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7751609004167127,"sd":0.003142873595741245,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7522017305471997,"sd":0.005519035037540703,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7598416672131473,"sd":0.0026258184520270263,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7808530973297696,"sd":0.004502442969436131,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7334018531229722,"sd":0.002786687515645859,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7256385694987327,"sd":0.0006283424867495947,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7638608608133874,"sd":0.0009214146253958092,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7741590450954007,"sd":0.006187734731751329,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7859259571833884,"sd":0.00019207625759468192,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7426374633028441,"sd":0.0020018143592024865,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":0.6987851286602618,"sd":0.002207707239288421,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rinalmo","mean":0.7667017285267144,"sd":0.00425853773712214,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.27444091521211894,"sd":0.004368946307479616,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.737027911010853,"sd":0.00027245904297538927,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rnafm","mean":0.70516072013641,"sd":0.002049173801167827,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"splicebert","mean":0.7181453691198239,"sd":0.0021327772390436605,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false}]},{"id":"bulk_rna_expression_256","benchmark":"ours_regression","name":"Gene Expression Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.7870501615642387,"sd":0.003021137596020311,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.687880882544723,"sd":0.00410339264932829,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"baseline_lstm","mean":0.7017165781280967,"sd":0.0008201749901697091,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"borzoi","mean":0.7565497166061057,"sd":0.0027093343214230356,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7066406319317498,"sd":0.0018168224440168858,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6878593086061271,"sd":0.018331589402513913,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6954833511422235,"sd":0.000857045229273148,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.7188792545338176,"sd":0.008217199990025303,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"dnabert-4","mean":0.7197379351495237,"sd":0.0034609535000018317,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-5","mean":0.7226783474551085,"sd":0.0037095892152129166,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-6","mean":0.7285500009005087,"sd":0.0024185097600838744,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert2","mean":0.6963698977510401,"sd":0.005740589439012231,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"enformer","mean":0.7587149991759816,"sd":0.005484302321809959,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7032104560897364,"sd":0.006764537058337265,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.690853582562557,"sd":0.0031202338279841947,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7189410648143797,"sd":0.0003850258187066036,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6856022198139965,"sd":0.0046085736683887666,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6868248760986521,"sd":0.003083463832112234,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7320294103064668,"sd":0.0033814989889347264,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7120100689548581,"sd":0.00456412612656849,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7477396823981995,"sd":0.0020996813936568163,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.7309135773502458,"sd":0.004140917507842276,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.713657435409099,"sd":0.006368089843736655,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7104114824325894,"sd":0.00203842902121586,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7038234246413339,"sd":0.0053389585931724525,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6600926700465121,"sd":0.013270813721836886,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7216518857617681,"sd":0.0030761709009326254,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7282916246245177,"sd":0.005061094818563012,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7154026627178519,"sd":0.0018633147721761499,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.717236608909337,"sd":0.003651570023683259,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7321818896373289,"sd":0.0017109500867552273,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6999156840416185,"sd":0.0011098767870318675,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6921763730472581,"sd":0.009157653734945492,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.712256040160202,"sd":0.0021684575735339672,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7300519599742997,"sd":0.006810917961046033,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7355308205958333,"sd":0.003619348694425524,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7086789272651803,"sd":0.004100286530158778,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"orthrus-4track","mean":0.7012781415652352,"sd":0.0013841383878224678,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.7221783369419666,"sd":0.009896645966264517,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"rnabert","mean":0.6361774127902785,"sd":0.003175451590688031,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7064294826494047,"sd":0.0031192387964448787,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnafm","mean":0.70849599268405,"sd":0.0008572561669124393,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"splicebert","mean":0.737492517107093,"sd":0.0036331069591264338,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false}]},{"id":"bulk_rna_expression_512","benchmark":"ours_regression","name":"Gene Expression Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8110624444279415,"sd":0.0016294639533594507,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6982669379436457,"sd":0.008505547200678306,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.71423450617865,"sd":0.0025139956737668346,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.7712897817694169,"sd":0.0015716400505374935,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7217548088301194,"sd":0.0018574893989788174,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7260304228170765,"sd":0.002767242447792691,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.708747957081504,"sd":0.00027055066244424105,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-3","mean":0.7351497089876664,"sd":0.004033914614849434,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-4","mean":0.7412023438796251,"sd":0.0044797681272586345,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-5","mean":0.7412501461441848,"sd":0.008155179330073923,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert-6","mean":0.7331369918038145,"sd":0.006637027233786211,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"dnabert2","mean":0.7267797062458914,"sd":0.0013925521360113678,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.7527198477100266,"sd":0.0036723336670853777,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"evo2_1b_base","mean":0.699968619968914,"sd":0.007649844357818968,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7065327538346593,"sd":0.0036058003318137927,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7350064137940212,"sd":0.002296746351835338,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7019902454947384,"sd":0.002698462257460607,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7035955512856567,"sd":0.001788480703190076,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7452957651940627,"sd":0.003584820238749935,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7343815334161323,"sd":0.00459902757316215,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7640348884401512,"sd":0.0039776361667372825,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.7454764897114733,"sd":0.0017151340362638093,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.7273297741436373,"sd":0.00039034226740457256,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7269757079180268,"sd":0.007637969082708872,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7212639779395539,"sd":0.0038774911949608477,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.675007857075776,"sd":0.0013021169231499216,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7391606785411549,"sd":0.006555727564038026,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7487189967330645,"sd":0.004109248427877728,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7403301180852003,"sd":0.0025596443547496466,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7330419999401845,"sd":0.0020587985651693286,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7509690072975709,"sd":0.0010809876163544957,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7106863063788725,"sd":0.004157611974762958,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7178742933841918,"sd":0.0017151976572877185,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7298394472043103,"sd":0.0036663844712034412,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7365967719536363,"sd":0.0059501932049737995,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7523783381652768,"sd":0.002077485774074525,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7177550706250507,"sd":0.001768804524398066,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"orthrus-4track","mean":0.7110431900811411,"sd":0.0015310961459738444,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.741261098777584,"sd":0.004105159975287967,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"rnabert","mean":0.6389468586263165,"sd":0.003540019999903328,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7248857990013421,"sd":0.0023726949620901977,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnafm","mean":0.7297737279189994,"sd":0.007736181089296782,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.7514623098388532,"sd":0.0040140130947571035,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false}]},{"id":"cov_vaccine_degradation","benchmark":"ours_regression","name":"SARS-CoV-2 Vaccine Degradation Prediction","modality":"RNA","metric":"spearman","protocol_variant":"grouped_or_chronological","results":[{"model":"alphagenome","mean":-0.11635336844140957,"sd":0.046198130055358555,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"baseline_cnn","mean":-0.06133407935494875,"sd":0.10273271464886298,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"baseline_lstm","mean":-0.11878579629089045,"sd":0.03684888065608041,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.044440869914666434,"sd":0.09135414468268428,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.1819300954348599,"sd":0.07441458953988812,"seeds":[1,2,4],"note":20,"rank":2.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.08639780941913122,"sd":0.11543610824089577,"seeds":[1,2,4],"note":0,"rank":8.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.010765910844984303,"sd":0.10455058763981448,"seeds":[1,3,4],"note":0,"rank":16.0,"top_score":false},{"model":"dnabert-3","mean":-0.03911627787999006,"sd":0.01210951642145532,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"dnabert-4","mean":-0.08391911423602799,"sd":0.08065259348773463,"seeds":[1,3,4],"note":0,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":-0.041030794100554685,"sd":0.1333694574127848,"seeds":[1,2,4],"note":0,"rank":25.0,"top_score":false},{"model":"dnabert-6","mean":-0.14383315487313553,"sd":0.009731354597175296,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert2","mean":0.047882632944033644,"sd":0.087675358654253,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"enformer","mean":0.20834690288617555,"sd":0.09695066577556136,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"evo2_1b_base","mean":0.06102019542099433,"sd":0.13733396752923233,"seeds":[1,3,4],"note":0,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base","mean":-0.1947711648241343,"sd":0.0776294601460167,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.003891156571254236,"sd":0.04463961047523391,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":-0.2833463009830081,"sd":0.031673855295692256,"seeds":[1,2,4],"note":null,"rank":43.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":-0.228198647998895,"sd":0.04500822318287143,"seeds":[1,2,6],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":-0.05863573179249946,"sd":0.08366247618730766,"seeds":[2,3,4],"note":0,"rank":26.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.13892339948433022,"sd":0.07155668173830435,"seeds":[1,3,5],"note":null,"rank":3.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":-0.03788820358359015,"sd":0.005115537433386675,"seeds":[2,3,4],"note":null,"rank":23.0,"top_score":false},{"model":"phylogpn","mean":-0.10528366664191881,"sd":0.1035610027328262,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"grover","mean":0.004694899714118782,"sd":0.07848361188093145,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":-0.07856504100649749,"sd":0.08141760962359226,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":-0.13585160810162453,"sd":0.09318073563058889,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.1806621865165434,"sd":0.07182927668756173,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":-0.013455060490582598,"sd":0.10852894057477566,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv3-650m-pre","mean":-0.11742916354619085,"sd":0.09434980607792712,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-8m-pre","mean":-0.07805896328962426,"sd":0.09259481835420508,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":-0.02556229402388991,"sd":0.03471307990911747,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":-0.014145209514937467,"sd":0.03887435756100946,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.01957836756305013,"sd":0.08967249378707892,"seeds":[1,3,4],"note":0,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":-0.06601707800404923,"sd":0.062692714041539,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.0942133069957918,"sd":0.017566370890107062,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.020538827703203075,"sd":0.09343853564115859,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":-0.09130854912427258,"sd":0.034754305373455883,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.008538354658169012,"sd":0.035831347358277206,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"orthrus-4track","mean":0.02916264514053406,"sd":0.13262426656281429,"seeds":[1,2,4],"note":0,"rank":13.0,"top_score":false},{"model":"rinalmo","mean":0.08841724048065486,"sd":0.06443964909962012,"seeds":[3,6,7],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.10145018867404548,"sd":0.10419307776598888,"seeds":[1,3,5],"note":0,"rank":5.0,"top_score":false},{"model":"rnaernie","mean":0.11719565756080183,"sd":0.07383844605527287,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnafm","mean":0.0328660874886706,"sd":0.08537263038072251,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"splicebert","mean":-0.07688541906040951,"sd":0.09215951557922034,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false}]},{"id":"cov_vaccine_degradation_random_iid","benchmark":"ours_regression","name":"SARS-CoV-2 Vaccine Degradation Prediction","modality":"RNA","metric":"spearman","protocol_variant":"random_iid","results":[{"model":"alphagenome","mean":0.7120036221452705,"sd":0.012236961789237487,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"baseline_cnn","mean":0.6022553202185946,"sd":0.0347032746356229,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"baseline_lstm","mean":0.6040741109083155,"sd":0.020223382515615805,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.6764231861273007,"sd":0.030471168511995505,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6420521123912472,"sd":0.03136473934551433,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6916792652801957,"sd":0.015968342279336294,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6555309803905094,"sd":0.02467051385872775,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-3","mean":0.6818986192361431,"sd":0.025871235577840775,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert-4","mean":0.6840685100521041,"sd":0.007098497511438096,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-5","mean":0.6823349953891292,"sd":0.026014471262330568,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-6","mean":0.6303981623655734,"sd":0.011581055771203528,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert2","mean":0.6441949203467033,"sd":0.0355594843920629,"seeds":[1,3,5],"note":0,"rank":22.0,"top_score":false},{"model":"enformer","mean":0.6043173222162552,"sd":0.02569064569890105,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7348498216813296,"sd":0.017593100224893306,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5956930722443462,"sd":0.02846664749534933,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6329412708870644,"sd":0.013303245347287407,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.4992887442625386,"sd":0.03305386152348536,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.4788115568132416,"sd":0.055475889688538914,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6337475300419297,"sd":0.04271089487811139,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6364703380713059,"sd":0.02852045794209688,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6312324450308541,"sd":0.04116657594346766,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"phylogpn","mean":0.5109408218233149,"sd":0.03866697215721887,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"grover","mean":0.6863192197368088,"sd":0.010319482606413397,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6114771097214522,"sd":0.01607001209093675,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6246813904402196,"sd":0.0323408075869267,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5988943464864566,"sd":0.012160955694502382,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7205631773494934,"sd":0.013817609326054518,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7372038125169662,"sd":0.017576940174056713,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6804430901935943,"sd":0.01138037751367635,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6403218386522009,"sd":0.021822064993135448,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6753529338492458,"sd":0.00821646057188734,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6684708922302214,"sd":0.006886569630741286,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6585950691079245,"sd":0.008800876432664648,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6688820944908137,"sd":0.05233879294464391,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6356126865533306,"sd":0.012290135842341326,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6811774319487034,"sd":0.020005824316847224,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6624460815528315,"sd":0.007409346255627989,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":0.5802069211894808,"sd":0.02413143658184439,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rinalmo","mean":0.7440983915500665,"sd":0.02042517106969094,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"rnabert","mean":0.6375931376799346,"sd":0.029833218135299492,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnaernie","mean":0.7017600715089698,"sd":0.006916029125001576,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnafm","mean":0.7208453424040089,"sd":0.038690640467773096,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"splicebert","mean":0.6424840375838233,"sd":0.012173528728450643,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false}]},{"id":"enhancer_activity_deepstarr_hct116","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8420310789960599,"sd":0.0018216276429195524,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.3630413941278518,"sd":0.040296884470497146,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"baseline_lstm","mean":0.2148334635571788,"sd":0.01146491881725417,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"borzoi","mean":0.7883293209307668,"sd":0.012524167927696506,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5669097820298878,"sd":0.06229932218410842,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5320222481961394,"sd":0.03554885695752162,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.21720136466699796,"sd":0.08034448305452845,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.5637384845653722,"sd":0.013043572045409848,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"dnabert-4","mean":0.594268481381446,"sd":0.0047767708312458305,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-5","mean":0.5813726796199367,"sd":0.007309863622084428,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert-6","mean":0.5877137235896013,"sd":0.006844912719540852,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert2","mean":0.44300530045747805,"sd":0.012368888574221961,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"enformer","mean":0.740007301157757,"sd":0.003011515846846122,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5497918949723741,"sd":0.03720151841072618,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5261339704440465,"sd":0.0067349165986344,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5524468914748578,"sd":0.009098003429829641,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5063294366043107,"sd":0.025631128898548947,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.48512303040469,"sd":0.011178899297091255,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5886944531497805,"sd":0.0046093488447398814,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5862499411149321,"sd":0.017902337752047665,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6552131058207735,"sd":0.014854987766267253,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.439879522752658,"sd":0.030654997247859285,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.5669226279310917,"sd":0.003866075335125697,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"hyena_large_1m","mean":0.4307485981226556,"sd":0.07120157510067167,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4393514503929189,"sd":0.037916543952458276,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.286770186654194,"sd":0.01286064233047205,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6098058493736577,"sd":0.020185197664883855,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.710458864761675,"sd":0.008312506293630046,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5439773066800668,"sd":0.024985461215688218,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.498450896026291,"sd":0.014269593874980462,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6233130332846396,"sd":0.0035908342548759014,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.42963159499547776,"sd":0.0026600905436501574,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4403556313521326,"sd":0.005597407989061632,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.4522053149720391,"sd":0.03303927726812389,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5437210460403237,"sd":0.011117396764026248,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6261748659819713,"sd":0.008919540384087309,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.4175417091424512,"sd":0.012719751379366378,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"orthrus-4track","mean":0.2852682303816677,"sd":0.020934109002440776,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rinalmo","mean":0.5931514532020568,"sd":0.023817552497204755,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.11078675169815205,"sd":0.08126973669579458,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.5781742358214716,"sd":0.0105225758729503,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.5406989878859102,"sd":0.00887757745588706,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.5151073846752647,"sd":0.011912060005177976,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false}]},{"id":"enhancer_activity_hct116_brd2","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.49184596127864033,"sd":0.012061831075438716,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.006816651701278076,"sd":0.03726464460029321,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.07169274971998303,"sd":0.011606087125887184,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.42246982380890313,"sd":0.010117520690723927,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.1356024500459856,"sd":0.07998389455584987,"seeds":[1,3,4],"note":0,"rank":31.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.20775205519757758,"sd":0.045679365686400436,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"cnn-56m-random-init","mean":-0.004614984806476546,"sd":0.02106545747373013,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.09773987711165377,"sd":0.034040418916194824,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.10374193005089592,"sd":0.008136824938175259,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-5","mean":0.12472857616242038,"sd":0.005551832965071984,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.08615768183154228,"sd":0.0054789305761270115,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.16637616052771578,"sd":0.06513517107906008,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.35516660193158683,"sd":0.03106993703276593,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.005551028686306859,"sd":0.028872978516042563,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.1907108923974213,"sd":0.026055585387862,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.20505334223774493,"sd":0.01694716404924122,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.13402674964311345,"sd":0.03886222862761801,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.1273045950842559,"sd":0.033677481304851056,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.22101291088781083,"sd":0.014944053136032306,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.23954734797736255,"sd":0.007055241243559624,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.18596468972165536,"sd":0.046113945015685345,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"phylogpn","mean":0.23269741196891097,"sd":0.028779979927898908,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"grover","mean":0.18842062306699833,"sd":0.03200192277573765,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.16041791541283804,"sd":0.00800067959346378,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.16525739718293117,"sd":0.006916258633168407,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.05841057088333722,"sd":0.022671817576197566,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.19822525853105905,"sd":0.03167641804759835,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.19387834287044375,"sd":0.02318799929536741,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.2385230621934535,"sd":0.013696072199925385,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2456106991739141,"sd":0.014199336527918788,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.21556402936605737,"sd":0.027367946233359693,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.17845386143290248,"sd":0.013618320993533056,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.20696746240814035,"sd":0.04634488312512728,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.2097637592517022,"sd":0.02534249909020875,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.2125941553811526,"sd":0.036543538266527516,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.22146906398936117,"sd":0.030842604779564433,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.2521971251820889,"sd":0.00944433067930731,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"orthrus-4track","mean":0.14779236423346948,"sd":0.02289882937363724,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.2275809493092312,"sd":0.02502349932128267,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.04110702733179567,"sd":0.011421375017164481,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.144869515055058,"sd":0.031160731953695727,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rnafm","mean":0.1941972080977319,"sd":0.02126151648607341,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.2181945636183852,"sd":0.02011730203259461,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false}]},{"id":"enhancer_activity_hct116_brd4","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5492528482148651,"sd":0.0027497207749686875,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.02149490919234333,"sd":0.017246560123562276,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.21607261481943524,"sd":0.010700563838796659,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"borzoi","mean":0.4492193556266055,"sd":0.03051406692895311,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.23645112176821856,"sd":0.08205291323657023,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.24566974272792133,"sd":0.061056162311670846,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"cnn-56m-random-init","mean":-0.005320182170058286,"sd":0.09372175486784491,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.13474027296067242,"sd":0.02477128767000929,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-4","mean":0.12702635108683252,"sd":0.015241404636635762,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.1515200047184899,"sd":0.02916533144929811,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-6","mean":0.18686849768504443,"sd":0.020090683635892375,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert2","mean":0.23586784993169616,"sd":0.03082078473876387,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"enformer","mean":0.39522787809601617,"sd":0.03079475374210357,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.26563133397171,"sd":0.012822196487822426,"seeds":[2,3,4],"note":0,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.25636198751226263,"sd":0.02914931546206805,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.28539819841756,"sd":0.019198905880308606,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.21498411900639888,"sd":0.07976653683249478,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.21042536137734716,"sd":0.05475693340572288,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.24664678698553688,"sd":0.06154996752976971,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.26952461191534693,"sd":0.007145789595204376,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.15765773232505326,"sd":0.03970293914839307,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"phylogpn","mean":0.21394101379625163,"sd":0.016391685282686138,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"grover","mean":0.2868157545467482,"sd":0.01391125484709012,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"hyena_large_1m","mean":0.28187628123321984,"sd":0.027098952235422486,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.25021689205929315,"sd":0.011542071025025889,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.11446349298870956,"sd":0.02066301602273672,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.2808538542434974,"sd":0.023452639973089027,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.2838923271856579,"sd":0.009694146379438344,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.28814919458461236,"sd":0.016752483025446904,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2911024774388217,"sd":0.041292460364975074,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.19214297079961137,"sd":0.02669962048072953,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.24106456774241677,"sd":0.01252028396266635,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.23451780668747718,"sd":0.028358948421804663,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.2838887582301895,"sd":0.001144182353168963,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.31424425026766095,"sd":0.008748789835313582,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.2671609722823152,"sd":0.03856371731643797,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3133239166398146,"sd":0.028374861221286275,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"orthrus-4track","mean":0.21081093043815655,"sd":0.029872712085920284,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.30976958943398925,"sd":0.010660577559481183,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.027217031821581805,"sd":0.03463361634039305,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.24129449898221952,"sd":0.01588221584727207,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.25606551544468636,"sd":0.05399290291559673,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.2772696270063015,"sd":0.025819536003697165,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false}]},{"id":"enhancer_activity_hct116_cdk7","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.3725040317611286,"sd":0.004747138557019462,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.02388371221548494,"sd":0.032988986147469514,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"baseline_lstm","mean":0.04895086340929458,"sd":0.07852932344026241,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.28472745076819744,"sd":0.019157142945834277,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.06084684868938499,"sd":0.06510403202318978,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.07538792897838456,"sd":0.03002748617114916,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.018658454074878768,"sd":0.015565972616427164,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.11463553109277048,"sd":0.01501577238695009,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"dnabert-4","mean":0.11490350557444368,"sd":0.03060590893600111,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-5","mean":0.09559705810790722,"sd":0.009820451385452586,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert-6","mean":0.11058818145200557,"sd":0.017494418472782793,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"dnabert2","mean":0.15485977860497602,"sd":0.015052981170989646,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"enformer","mean":0.24281806373889187,"sd":0.018506468341356982,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.00940445728969558,"sd":0.07235086665425212,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.09523386296237482,"sd":0.028399004449878085,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.1613162008246097,"sd":0.03507299431856968,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.1323122622073847,"sd":0.03676431997574002,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.11007133253746229,"sd":0.016428095249489075,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.18210546347353446,"sd":0.010703591940736723,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.1377590311964653,"sd":0.02473733026355874,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.1624597575043383,"sd":0.038632248810518355,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"phylogpn","mean":0.1751308620480973,"sd":0.0211485826918093,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"grover","mean":0.0790443688359391,"sd":0.030307698552816526,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_large_1m","mean":0.08467759604715927,"sd":0.024877169503734836,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.015049409642065517,"sd":0.07799358125800869,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.04301009997961969,"sd":0.07750511905143363,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.15400970740332023,"sd":0.01697801438623625,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.18518719679862344,"sd":0.024093473000978124,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.11283833501168057,"sd":0.010788767243541456,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.1697512170368594,"sd":0.013148967049726705,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.2072443406901656,"sd":0.01629946826816419,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.11429713945973513,"sd":0.0025229091149429014,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.1389170595072762,"sd":0.02914800883334887,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.15459047690021013,"sd":0.037883378602546786,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.17094297300823064,"sd":0.013590085665865403,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.15781669555229394,"sd":0.023508842764283517,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.1298942766750225,"sd":0.006771001768102653,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.07457089546941312,"sd":0.06546150690387428,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.16229621641487757,"sd":0.037290854209978526,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"rnabert","mean":-0.05594149016418481,"sd":0.01836365021235544,"seeds":[1,3,4],"note":21,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.11051693071045439,"sd":0.023661071083297614,"seeds":[1,5,6],"note":22,"rank":27.0,"top_score":false},{"model":"rnafm","mean":0.08343011998755281,"sd":0.03780111602343554,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"splicebert","mean":0.15816305636929054,"sd":0.012322360897908616,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false}]},{"id":"enhancer_activity_hct116_cdk9","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.24982676082562055,"sd":0.008011924359173259,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.07032375327181059,"sd":0.04767688967387107,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"baseline_lstm","mean":0.06304396925664565,"sd":0.0027160240158911027,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.17265745182786899,"sd":0.0315747409927938,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.05829996996351245,"sd":0.029821954782024717,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.08882830294964525,"sd":0.011491976252337683,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.0636478480528567,"sd":0.004120123936040442,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-3","mean":0.141938308489092,"sd":0.013864845521513505,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-4","mean":0.170018983106144,"sd":0.023924862496106262,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"dnabert-5","mean":0.16956356579774734,"sd":0.03034024910520678,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-6","mean":0.11244133447421888,"sd":0.01006988240832344,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"dnabert2","mean":0.04838236438995983,"sd":0.029570093189960428,"seeds":[1,2,3],"note":23,"rank":43.0,"top_score":false},{"model":"enformer","mean":0.2169344938790063,"sd":0.006804927597877941,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"evo2_1b_base","mean":0.12321017839108529,"sd":0.034696235466365855,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.07693179448848698,"sd":0.011187492234149601,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.13643652448597465,"sd":0.012352392347450725,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.09671728802932934,"sd":0.008275494719657148,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.12526763266631663,"sd":0.01405220444287896,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.14069348856645184,"sd":0.010176593881709151,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.12578853372275145,"sd":0.002947074335328488,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.12130306078117029,"sd":0.016819268455222226,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"phylogpn","mean":0.1220545105520054,"sd":0.003403443288220957,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"grover","mean":0.13975282483304932,"sd":0.006115352538627781,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"hyena_large_1m","mean":0.07418763071447268,"sd":0.0026744686259252966,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.10598191035105226,"sd":0.01621464177091041,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.09764134875496126,"sd":0.02499157580352748,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.1337849387738877,"sd":0.021813796552426216,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.1324619612210321,"sd":0.020794818301686783,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.11717070921456106,"sd":0.018422808918805402,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.12923053229114886,"sd":0.015923183560877147,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.12824726272325157,"sd":0.011245689674859739,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.13239926977868863,"sd":0.022168476652123757,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.10192411905008773,"sd":0.022524539716721286,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.1237480201156054,"sd":0.01413397210435794,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.1123500335505528,"sd":0.00697256302411949,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.12188274780375674,"sd":0.009913438585903725,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.11795024024019711,"sd":0.010893615680487155,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"orthrus-4track","mean":0.11814295862360515,"sd":0.027957544829338904,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"rinalmo","mean":0.14000128287580477,"sd":0.008304899599459474,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.05196079917257019,"sd":0.01297903809813002,"seeds":[3,4,9],"note":24,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.10701166786483222,"sd":0.017956378153396554,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnafm","mean":0.06281274195242302,"sd":0.013232206630197377,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"splicebert","mean":0.073836723613533,"sd":0.047307143268714186,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false}]},{"id":"enhancer_activity_hct116_med14","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5059013906395665,"sd":0.011903695795315555,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.04254738695364871,"sd":0.03169148529901988,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.1080551424299569,"sd":0.041812043722169956,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.470869800300678,"sd":0.0147929092368418,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.07186033962449133,"sd":0.029310683384059552,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.14095734984295755,"sd":0.031248196014643286,"seeds":[2,3,4],"note":0,"rank":35.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.0697166403100831,"sd":0.004834360537562995,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.16876954961510432,"sd":0.04456708441607853,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-4","mean":0.1573819472233701,"sd":0.020369166546849716,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.1642763258510466,"sd":0.03240221367606895,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.1535147396896078,"sd":0.025680105239135147,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert2","mean":0.23100255127499347,"sd":0.01643388643666251,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"enformer","mean":0.39211236768608965,"sd":0.0023688370707860102,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.10619684683352984,"sd":0.06233654165591111,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.17078789076495376,"sd":0.029737066895162704,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.28194896524324065,"sd":0.019562748953514505,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.2305917562351857,"sd":0.008597700728798275,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.23672139453223698,"sd":0.015104108242778642,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.27054405316506186,"sd":0.013320711789092997,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2935189184564656,"sd":0.04533035412826586,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.2173672869744041,"sd":0.02396274182219484,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"phylogpn","mean":0.1867568077225203,"sd":0.008685488537004163,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"grover","mean":0.24956475959191735,"sd":0.01586607344805529,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"hyena_large_1m","mean":0.15570013905564098,"sd":0.030336652588484784,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.13883512048610275,"sd":0.010173457984189423,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.07754077031782858,"sd":0.032861934871204486,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.20166581124033572,"sd":0.07055654177828902,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.22673868126752802,"sd":0.02939592783419924,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.249165584049089,"sd":0.00936455654935975,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2568103418537143,"sd":0.021889751527791938,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.25633108713757313,"sd":0.012190871005962686,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.21613742750808487,"sd":0.010389752303463638,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.2264292788830749,"sd":0.034075503972229707,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.20781960471231112,"sd":0.04090003325527583,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.267655883292198,"sd":0.010315697793576535,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.21045460390835374,"sd":0.014619483027812151,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.2309161589230595,"sd":0.02103852365170934,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"orthrus-4track","mean":0.21365352716466396,"sd":0.022287441211326482,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rinalmo","mean":0.24130236702516092,"sd":0.03218243361204176,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"rnabert","mean":0.023985368085609665,"sd":0.026330571289677994,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.17004480763242716,"sd":0.019878993778028962,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rnafm","mean":0.19608340308297248,"sd":0.022955502394233062,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"splicebert","mean":0.23295934460440107,"sd":0.010736126309026019,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false}]},{"id":"enhancer_activity_hct116_p300cbp","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6462855866299844,"sd":0.002513204614814921,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":-0.02194812653653327,"sd":0.01440943252844614,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"baseline_lstm","mean":0.17141609560477664,"sd":0.0370844355363641,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.5913036706990876,"sd":0.011740615374463529,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.18464168050279572,"sd":0.1192845249848452,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.01680975636028425,"sd":0.03502285883941568,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.05184186144356057,"sd":0.01586473131330125,"seeds":[1,2,3],"note":25,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.23286101640288825,"sd":0.018056759973537957,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.2597188309995874,"sd":0.0031129746761273547,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.2047707177796374,"sd":0.01383863159313423,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-6","mean":0.2391140685048314,"sd":0.01665472535204989,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert2","mean":0.3313378079281923,"sd":0.06815689994484413,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"enformer","mean":0.566426417675889,"sd":0.017328594324791634,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3475713208868258,"sd":0.07480345399389389,"seeds":[3,5,7],"note":0,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.2736600524707941,"sd":0.028238224976089583,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.3668778764566716,"sd":0.032666362162859106,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.33793567731209656,"sd":0.04148016492746619,"seeds":[1,2,4],"note":0,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.23627757965818172,"sd":0.07272801250143396,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.4092426039526337,"sd":0.02333214234351466,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4294532207450817,"sd":0.01866855587944273,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.37534292346120574,"sd":0.06739882411340233,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"phylogpn","mean":0.3227552383381429,"sd":0.0222342146783097,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"grover","mean":0.40359715828433235,"sd":0.01657325461597684,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3037741566552572,"sd":0.011206129505427757,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3066262417654644,"sd":0.043531265077783515,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.07918008214218732,"sd":0.06719885013906327,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.3840957879379001,"sd":0.023310803242771107,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.4289001850315211,"sd":0.005640742341727601,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.42207578071432716,"sd":0.025233920754026966,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.3889011048566955,"sd":0.019972174856721835,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.4215726807692093,"sd":0.010416055058359509,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.31804860933037615,"sd":0.016781900181049254,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3451066283674161,"sd":0.006990066057572224,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.4034901804844027,"sd":0.009834900435604197,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.4011598998722393,"sd":0.014027296193543764,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.39633811455314005,"sd":0.023996413595724252,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.41885531289408606,"sd":0.001588038733949048,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"orthrus-4track","mean":0.2600304723803549,"sd":0.026164920454545562,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"rinalmo","mean":0.40425435374290103,"sd":0.055619552027817945,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.052601852954059766,"sd":0.03569159442067901,"seeds":[1,2,3],"note":26,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.35444237458644595,"sd":0.01636722532159151,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"rnafm","mean":0.3715163720254229,"sd":0.049337887956830605,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"splicebert","mean":0.366998143068821,"sd":0.01620868110927203,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false}]},{"id":"enhancer_activity_hct116_parental","benchmark":"ours_regression","name":"Enhancer Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.13314743571114218,"sd":0.0058577460801591775,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"baseline_cnn","mean":0.041828446482989515,"sd":0.05652276642203918,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.014752291449505217,"sd":0.006225098381795819,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.07742166198492086,"sd":0.07220023864494482,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.04756291211947073,"sd":0.01819562556180427,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.05526735120450995,"sd":0.08779034923125201,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"cnn-56m-random-init","mean":-0.0011379121415001445,"sd":0.00800431122138187,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.08770084141678224,"sd":0.025604435407243546,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"dnabert-4","mean":0.10938172928612068,"sd":0.038466473667950495,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert-5","mean":0.08205690613380245,"sd":0.039583557624187034,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-6","mean":0.09806001935330422,"sd":0.03786280008572241,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert2","mean":0.010887078311364082,"sd":0.037200320751753016,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"enformer","mean":0.10706654169534802,"sd":0.018996581264501076,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"evo2_1b_base","mean":-0.012187993243441008,"sd":0.03370224966997809,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.06523039946663102,"sd":0.0294762180247657,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.11381655204728278,"sd":0.0177994363453101,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.03534741016900424,"sd":0.020673140111643364,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.051581374986966695,"sd":0.036282467543168884,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.11185785163068593,"sd":0.05247998531728917,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.056899148118130974,"sd":0.016912183968827623,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.11759403991814357,"sd":0.029013610111317475,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"phylogpn","mean":0.1265718354965817,"sd":0.007785207271785554,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"grover","mean":0.07703512921687752,"sd":0.04148834964407231,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.08584678971646004,"sd":0.06589296808604428,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.046776475027666485,"sd":0.020803182738072398,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.005437962531604757,"sd":0.018671181280555865,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.10286626608776754,"sd":0.03290479637489123,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.09929384413757592,"sd":0.03628032306005285,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.06236730680304617,"sd":0.029462454576712184,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.11199178741928495,"sd":0.018375730001845416,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.10672231026667807,"sd":0.015001888488524977,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.10154485634772399,"sd":0.05119053641397828,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.016653107086832118,"sd":0.032688928592202564,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.12883747398005108,"sd":0.0439783655645523,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.10993160937018832,"sd":0.015072019834237258,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.11142890725861103,"sd":0.006758932751507791,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.1228288267542246,"sd":0.03898836655148894,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"orthrus-4track","mean":0.05410444289777572,"sd":0.022357713743735127,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rinalmo","mean":0.022152931363467587,"sd":0.046381186123676255,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rnabert","mean":0.014267487419462736,"sd":0.035540927518094595,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"rnaernie","mean":0.05424948010260803,"sd":0.02100711214886174,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rnafm","mean":0.13638378176046778,"sd":0.018393833330877535,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"splicebert","mean":0.10593337027118553,"sd":0.004614807987054512,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false}]},{"id":"fluorescence_amacgfp","benchmark":"ours_regression","name":"Fluorescence Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8323911046837761,"sd":0.012555609794006717,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"baseline_cnn","mean":0.7741935546304358,"sd":0.005926853843470268,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"baseline_lstm","mean":0.07204542070859349,"sd":0.016504723974435496,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.8718764749750484,"sd":0.0035163602508839036,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.005011656460994388,"sd":0.0030051298844893624,"seeds":[1,2,3],"note":27,"rank":41.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":-0.016603941548553346,"sd":0.03738595718550167,"seeds":[2,3,4],"note":15,"rank":42.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.10948746088142072,"sd":0.02504713648817409,"seeds":[1,2,3],"note":28,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.3915448723116177,"sd":0.06184912505087666,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-4","mean":0.42061422449139213,"sd":0.07322893580519761,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-5","mean":0.44678329231369074,"sd":0.027243430772725365,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-6","mean":0.527108127771256,"sd":0.04495752002477628,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert2","mean":0.40954440427905764,"sd":0.047430022403119505,"seeds":[1,3,5],"note":null,"rank":32.0,"top_score":false},{"model":"enformer","mean":0.7818188289021646,"sd":0.01181075556292029,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"evo2_1b_base","mean":0.1470589480190616,"sd":0.04246553008537411,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7410880375792286,"sd":0.04727200070433887,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8191267404950803,"sd":0.021460375695989076,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6456708050139093,"sd":0.04926024594107813,"seeds":[1,2,4],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6335975112797648,"sd":0.038927568500579206,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.8303487385357752,"sd":0.045546543460387015,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6869256639774429,"sd":0.009756006599641124,"seeds":[2,3,6],"note":29,"rank":24.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8642753385428493,"sd":0.0016816610403910348,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"phylogpn","mean":0.8620245036605695,"sd":0.003812442442963349,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.8177778831012378,"sd":0.033763417369110225,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"hyena_large_1m","mean":0.08764603049391913,"sd":0.07069985332763962,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.025670221274064025,"sd":0.011306127086566163,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.8788454004911633,"sd":0.0017933171493511864,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8193506260039145,"sd":0.006740702852054957,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7463967993190895,"sd":0.05176673542244001,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7868667784314379,"sd":0.009628414572492687,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.8929332667407284,"sd":0.002099050372772822,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8878346558806852,"sd":0.005697144591622844,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8815710754762957,"sd":0.002683064697904845,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.8789086985340466,"sd":0.004408927560022744,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7945194885057504,"sd":0.019693431810572686,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.8286568969603548,"sd":0.0032916347750889087,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8623630195306166,"sd":0.003578079136185658,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8318092192677274,"sd":0.0029707041886784832,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.5299567455789206,"sd":0.04866223033592591,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"rinalmo","mean":0.7892624187672075,"sd":0.04432487474860777,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"rnabert","mean":0.1093780381978056,"sd":0.04257328020136786,"seeds":[1,3,4],"note":30,"rank":36.0,"top_score":false},{"model":"rnaernie","mean":0.4867299372709057,"sd":0.08331595195632982,"seeds":[2,3,4],"note":0,"rank":29.0,"top_score":false},{"model":"rnafm","mean":0.04734098520345213,"sd":0.07226313316577057,"seeds":[1,2,4],"note":15,"rank":39.0,"top_score":false},{"model":"splicebert","mean":-0.036489281546209416,"sd":0.02679533099112749,"seeds":[1,2,4],"note":15,"rank":43.0,"top_score":false}]},{"id":"fluorescence_avgfp","benchmark":"ours_regression","name":"Fluorescence Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6887489732023789,"sd":0.0013181671622238432,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"baseline_cnn","mean":0.6686245678715883,"sd":0.0030555534602806156,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"baseline_lstm","mean":0.6280550889404882,"sd":0.044686069236634825,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"borzoi","mean":0.6949718937367907,"sd":0.0011600276164103307,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.17225245148585203,"sd":0.007221253497623858,"seeds":[6,17,12],"note":31,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.04804550023114676,"sd":0.00628404761040331,"seeds":[1,2,3],"note":32,"rank":41.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.028293242107591376,"sd":0.020503042986697327,"seeds":[1,2,3],"note":28,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.560646867225345,"sd":0.005704004636412953,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.5715035811172576,"sd":0.009235271267424943,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-5","mean":0.573335824163145,"sd":0.008909458696135337,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert-6","mean":0.5955232842160693,"sd":0.012283711864891559,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert2","mean":0.483263977368927,"sd":0.1183881150662677,"seeds":[1,3,4],"note":15,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.678666132653977,"sd":0.004051990927111713,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"evo2_1b_base","mean":0.29544460576558673,"sd":0.055472333239727036,"seeds":[1,4,5],"note":0,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6532102503696894,"sd":0.010985227980831038,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6664399781213212,"sd":0.010699350062715001,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6180261463819363,"sd":0.0377128081305107,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6362451964873861,"sd":0.00478465637674305,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6753574681465134,"sd":0.0027330422601528565,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.632560475204377,"sd":0.009233534203574019,"seeds":[1,2,3],"note":33,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6935531031588851,"sd":0.00187959807702483,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.6916143152207229,"sd":0.0015180576384236496,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"grover","mean":0.6580449405913452,"sd":0.00892230909076922,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.35828031634652463,"sd":0.047348057913047824,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3096620043314217,"sd":0.010314430265533922,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6914592766389314,"sd":0.0003913269796761063,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6876178807172618,"sd":0.0006939280738413564,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.651459496879543,"sd":0.026356770919275064,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6718463728937022,"sd":0.006788955647052098,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6942884813448504,"sd":0.0020073682740189562,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6946915850661366,"sd":0.00047923968048659915,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6939009262353525,"sd":0.0025941877619681934,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6914220503006016,"sd":0.0034397981903996483,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6691552913380439,"sd":0.003007246970379015,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6860197554631174,"sd":0.003353788351966564,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6928484380802792,"sd":0.0024366990274733865,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6872782400180738,"sd":0.001050468917844963,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"orthrus-4track","mean":0.5555350378200907,"sd":0.010775286525847928,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rinalmo","mean":0.6604830598146926,"sd":0.023504400391638716,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"rnabert","mean":0.044150173298457024,"sd":0.04073050485578125,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.5610661953503658,"sd":0.09466038453891726,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"rnafm","mean":0.32854251563966835,"sd":0.04804459271941639,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"splicebert","mean":0.5297918749172698,"sd":0.032205989514696165,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false}]},{"id":"fluorescence_cgregfp","benchmark":"ours_regression","name":"Fluorescence Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5566898991955996,"sd":0.0003294184208874379,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"baseline_cnn","mean":0.5142839399568082,"sd":0.0038409616674496033,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"baseline_lstm","mean":0.1237515995433612,"sd":0.02600296817058619,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"borzoi","mean":0.5658769390812208,"sd":0.0027424870884446293,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.01332507592320602,"sd":0.009709530461302065,"seeds":[1,3,4],"note":34,"rank":43.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.049680781193363645,"sd":0.01393013299173626,"seeds":[1,2,3],"note":32,"rank":40.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.024856995436366124,"sd":0.03720450133259844,"seeds":[1,2,3],"note":28,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.2973026872474063,"sd":0.016238411541440897,"seeds":[2,3,4],"note":0,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.3592537780853289,"sd":0.01432038460070238,"seeds":[1,3,4],"note":0,"rank":29.0,"top_score":false},{"model":"dnabert-5","mean":0.3534287261890136,"sd":0.03613638722332284,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-6","mean":0.372009815097409,"sd":0.03732280789885272,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert2","mean":0.06293287161770443,"sd":0.1302140019476069,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"enformer","mean":0.5162164764705305,"sd":0.007578867354659742,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"evo2_1b_base","mean":0.1495604429185311,"sd":0.09685259677333344,"seeds":[1,3,4],"note":0,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4992862401832392,"sd":0.010403356606212057,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5282396762219154,"sd":0.004089217597013555,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.4142496191470942,"sd":0.011150116151079425,"seeds":[1,2,4],"note":0,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.4534890961110037,"sd":0.020767919933167962,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5314113101683183,"sd":0.004612836915311521,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5086537270784415,"sd":0.011211875841481237,"seeds":[3,4,5],"note":35,"rank":22.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5406701258207134,"sd":0.0012066882002810186,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"phylogpn","mean":0.5454649536309436,"sd":0.0016182250668593052,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"grover","mean":0.5259197823061553,"sd":0.01624836268625438,"seeds":[1,3,8],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":0.05420906449895287,"sd":0.0589888415998434,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.11575943344727502,"sd":0.1434176455902966,"seeds":[2,4,5],"note":0,"rank":35.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5622486795202012,"sd":0.00511450911931634,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5432228266966089,"sd":0.004467692626918038,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5100800928478578,"sd":0.033247492733858505,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5303456422452292,"sd":0.004265551444441445,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5571846709119653,"sd":0.002113182656710445,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.566334745030732,"sd":0.0015056857299146503,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-1000g","mean":0.559838423392547,"sd":0.0036706297837925937,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5562245881757516,"sd":0.0031481364149758264,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5288319096285613,"sd":0.015342665415869055,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.544092084567443,"sd":0.00882706224666672,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5541564794614071,"sd":0.01014551762896297,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.551755195188191,"sd":0.0005803973485211638,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"orthrus-4track","mean":0.33737890290071304,"sd":0.0369487384886849,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"rinalmo","mean":0.4253536048574493,"sd":0.07884250766183246,"seeds":[1,2,3],"note":36,"rank":25.0,"top_score":false},{"model":"rnabert","mean":0.015394462755529312,"sd":0.015393572041564843,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.39240198588151537,"sd":0.04273986752227882,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"rnafm","mean":0.05431212433711399,"sd":0.027926618127038613,"seeds":[1,4,5],"note":15,"rank":37.0,"top_score":false},{"model":"splicebert","mean":0.050934391005762025,"sd":0.04425645449850087,"seeds":[3,4,5],"note":15,"rank":39.0,"top_score":false}]},{"id":"fluorescence_pplugfp2","benchmark":"ours_regression","name":"Fluorescence Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8705185770397392,"sd":0.004839029212575876,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.8476743994147732,"sd":0.002381771255522447,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"baseline_lstm","mean":0.06196164719109022,"sd":0.008901022802794928,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"borzoi","mean":0.890760363324182,"sd":0.0007037993593606462,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.00433875468903233,"sd":0.0035409529355437367,"seeds":[5,7,11],"note":37,"rank":42.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":-0.02826989251135711,"sd":0.01821679995133225,"seeds":[1,2,3],"note":38,"rank":43.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.12657833073279048,"sd":0.023519183938270804,"seeds":[1,2,3],"note":28,"rank":36.0,"top_score":false},{"model":"dnabert-3","mean":0.5957008280862768,"sd":0.05068024585888749,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.5209589224004222,"sd":0.08637122659114553,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-5","mean":0.5987760487657218,"sd":0.06669606615828466,"seeds":[2,3,4],"note":0,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.669677935905519,"sd":0.00946956750308703,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert2","mean":0.6019632753814438,"sd":0.12236160336110466,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"enformer","mean":0.8442950815768597,"sd":0.0037670141167172964,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"evo2_1b_base","mean":0.12150885775646961,"sd":0.1130323708613739,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.8570794789596322,"sd":0.008112651908734664,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8689955285182975,"sd":0.001159417856509023,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.8224597199439719,"sd":0.007486093375751658,"seeds":[1,2,4],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7880685614570725,"sd":0.02364521064697837,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.8740398065048914,"sd":0.0016346421722484075,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7912477425315162,"sd":0.030700423542421897,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.886453698197764,"sd":0.0019576999078690974,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.8778111491849009,"sd":0.0012031069495832033,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.8572183585157234,"sd":0.004511382733096987,"seeds":[1,2,4],"note":null,"rank":17.0,"top_score":false},{"model":"hyena_large_1m","mean":0.007572931876308208,"sd":0.014182729856186408,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.13035431013854165,"sd":0.01916987730089875,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.878841766784336,"sd":0.0012080501945354665,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8680577286758165,"sd":0.004220749591230939,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.829478974442658,"sd":0.032316279118101275,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.8477088678561783,"sd":0.009954461433828617,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.8946412643132918,"sd":0.0017832992276000882,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8936487519490758,"sd":0.0026247595381763294,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8863234741240698,"sd":0.001147030671949541,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.888497473775382,"sd":0.00033346700827852706,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.8602469793664117,"sd":0.006431101276786704,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.8774720357224551,"sd":0.005066837173366052,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8837584382750531,"sd":0.003984307070728208,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8732783755229004,"sd":0.0015305379356138622,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"orthrus-4track","mean":0.6931141835360831,"sd":0.08225811735575903,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rinalmo","mean":0.8510960326357024,"sd":0.017549008968604413,"seeds":[1,2,4],"note":15,"rank":19.0,"top_score":false},{"model":"rnabert","mean":0.0120487012346989,"sd":0.01858285956040408,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.7897489351575026,"sd":0.05502994766873498,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnafm","mean":0.05287530893186013,"sd":0.08358402693642351,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"splicebert","mean":0.4497129716384612,"sd":0.08131065745857803,"seeds":[1,3,5],"note":null,"rank":34.0,"top_score":false}]},{"id":"hek_el","benchmark":"ours_regression","name":"Expression Level Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.12410495893946766,"sd":0.013721335766803577,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"baseline_cnn","mean":0.09855763292163591,"sd":0.047853773014515176,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"baseline_lstm","mean":0.1601640591616905,"sd":0.00953875243213055,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"borzoi","mean":0.11136244863844148,"sd":0.03300271000319108,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.07833687371032318,"sd":0.06061762905424101,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.1307503680886364,"sd":0.0087423765061607,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.056008947987543724,"sd":0.01832634364184269,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.15238241732285265,"sd":0.013786867280754218,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-4","mean":0.14867426463106306,"sd":0.03628119498829087,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-5","mean":0.18091597546219335,"sd":0.010650157869090831,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-6","mean":0.14430590725812034,"sd":0.012484588070004379,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"dnabert2","mean":0.1843005915800833,"sd":0.03220618336054591,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"enformer","mean":0.14146444060294833,"sd":0.024258999960875674,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"evo2_1b_base","mean":0.10232015510546841,"sd":0.04506240823093021,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.13390831016051513,"sd":0.021786246025056227,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.15454702951912097,"sd":0.004767116305707928,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.15281318692638907,"sd":0.014733252525952442,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.11301448305502602,"sd":0.05225280803043571,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.19111250761114706,"sd":0.008208452788971846,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.11955038697998009,"sd":0.00915361500847017,"seeds":[1,2,3],"note":39,"rank":31.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.10623145370467169,"sd":0.025411763538796584,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"phylogpn","mean":0.14955944233754895,"sd":0.026686638166932217,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"grover","mean":0.198427763136821,"sd":0.014196091685457629,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"hyena_large_1m","mean":0.0951464223531509,"sd":0.00938969137551306,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.13667340352303867,"sd":0.03093964396315382,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.015292530218044198,"sd":0.031406397871563005,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.15055796106459263,"sd":0.0065952184528925845,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.15107459857125613,"sd":0.02251654212275118,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.1473264504823141,"sd":0.019982600980250592,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.15646481092217288,"sd":0.02727175039482423,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.1627668608807828,"sd":0.013397965417962072,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.10739730226070231,"sd":0.041495430855750616,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.10755538531152102,"sd":0.052880522944952636,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.14504213006545097,"sd":0.011487284515602872,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.119919128287041,"sd":0.011078504786036891,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.15633624490774903,"sd":0.02425925061197382,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.16132248997117896,"sd":0.020951310828241934,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"orthrus-4track","mean":0.17679970227952582,"sd":0.040685100323094386,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rinalmo","mean":0.16168549041092434,"sd":0.01520175315566045,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.027270429408250797,"sd":0.007190587288221911,"seeds":[2,13,21],"note":40,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.15976864436250962,"sd":0.02725460031903696,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"rnafm","mean":0.16034123844871195,"sd":0.03409255052504919,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"splicebert","mean":0.18746300364285337,"sd":0.025226910311788316,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false}]},{"id":"hek_te","benchmark":"ours_regression","name":"Translation Efficiency Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.2695129165212648,"sd":0.02311268213258882,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.06201217847272832,"sd":0.03445143159970927,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.23702140229431667,"sd":0.03111377167189945,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"borzoi","mean":0.18922151208368393,"sd":0.038043332439744296,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.10918054959665549,"sd":0.09392503352913618,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.2355730178471335,"sd":0.008068461741176581,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.13016125141449647,"sd":0.008201356777299251,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.23452935339087463,"sd":0.02030203885352885,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"dnabert-4","mean":0.17615801915521154,"sd":0.02510179453444655,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert-5","mean":0.18448115366364,"sd":0.01198840439016766,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-6","mean":0.23065703389047998,"sd":0.013468474057951626,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert2","mean":0.1916318584439933,"sd":0.042869166823696216,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"enformer","mean":0.2161316017131942,"sd":0.014157889816990139,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"evo2_1b_base","mean":0.14212003668354425,"sd":0.016326199150008117,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.12210439979285599,"sd":0.053967502184392384,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.22329690215613246,"sd":0.024777483622916714,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.08781110186651785,"sd":0.03837161718761229,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.06616578744291646,"sd":0.08988602668908818,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.21555068111417564,"sd":0.010170067800288245,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2048173755557318,"sd":0.022135058254917787,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.2398105277703064,"sd":0.02351884242956131,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"phylogpn","mean":0.17457887200099217,"sd":0.02492607565934628,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"grover","mean":0.18191499079026,"sd":0.03803228316452396,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.24096946539788525,"sd":0.004551708648216054,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.21804118925581004,"sd":0.0073553925213555,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.03137198085992597,"sd":0.012278753395866102,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.2174767161860928,"sd":0.0063032694520792085,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.20860547442132513,"sd":0.026145616462866415,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.2423891307954475,"sd":0.021489849923161584,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.15538556307665002,"sd":0.03971091342932206,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.19388733931769683,"sd":0.0017171854032048482,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.13287894645750536,"sd":0.009792748047693232,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.17325433896660034,"sd":0.01637577791337243,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.1504931643352292,"sd":0.016533393110016524,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.15867846012812162,"sd":0.026543245391408,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.17342075227345138,"sd":0.0362114453740827,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.15878644659310182,"sd":0.03142145094307803,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"orthrus-4track","mean":0.1794489514871572,"sd":0.01930846428818166,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rinalmo","mean":0.21861917915057155,"sd":0.03299928894194544,"seeds":[1,2,3],"note":41,"rank":12.0,"top_score":false},{"model":"rnabert","mean":0.09648814578442273,"sd":0.014890611152135916,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rnaernie","mean":0.07138542140396723,"sd":0.07693112212152275,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnafm","mean":0.23341227974262177,"sd":0.014853154271480843,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"splicebert","mean":0.2460364257656506,"sd":0.015235456527178715,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false}]},{"id":"isoform","benchmark":"ours_regression","name":"APA Isoform Prediction","modality":"RNA","metric":"spearman","protocol_variant":"grouped_or_chronological","results":[{"model":"alphagenome","mean":0.42944703069791057,"sd":0.05901837592493915,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"baseline_cnn","mean":0.43413947297472333,"sd":0.011877469785003784,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"baseline_lstm","mean":-0.002336233993393252,"sd":0.13171445843479387,"seeds":[2,3,4],"note":0,"rank":35.0,"top_score":false},{"model":"borzoi","mean":0.6390181749770685,"sd":0.05336347124233198,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.410488802449589,"sd":0.023279894883314743,"seeds":[1,3,4],"note":0,"rank":20.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.22921481709873245,"sd":0.09246940048895778,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5548016317831829,"sd":0.08172760413508698,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"dnabert-3","mean":0.6108988374318376,"sd":0.008206130494321106,"seeds":[1,3,4],"note":null,"rank":2.0,"top_score":false},{"model":"dnabert-4","mean":0.5358185476342195,"sd":0.09435758312203203,"seeds":[1,3,4],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-5","mean":0.4523884701008055,"sd":0.03182215074514414,"seeds":[1,2,4],"note":0,"rank":14.0,"top_score":false},{"model":"dnabert-6","mean":0.4803444071597527,"sd":0.010392240190225709,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert2","mean":-0.20066328905181038,"sd":0.03429082711229303,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"enformer","mean":0.44319938198354786,"sd":0.024822543176087616,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"evo2_1b_base","mean":0.34101363468883034,"sd":0.12788278948032716,"seeds":[1,2,5],"note":0,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base","mean":-0.11703701503251839,"sd":0.018231864158584146,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":-0.04475621016624135,"sd":0.044860846005518484,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":-0.09767663965067623,"sd":0.03709633485122746,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":-0.10168042500321102,"sd":0.035657376870254616,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.17215482222040876,"sd":0.03485445302323359,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":-0.032251079516368276,"sd":0.19862980847928766,"seeds":[2,10,12],"note":35,"rank":37.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.43563907960548187,"sd":0.07679195820716347,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"phylogpn","mean":0.3578766120487094,"sd":0.023745140868591016,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"grover","mean":0.09270109572304107,"sd":0.03757381435314712,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"hyena_large_1m","mean":0.22409751475113182,"sd":0.07314667270273266,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.259201593999736,"sd":0.04864214291970423,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.0288203607342104,"sd":0.003993555384658704,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.22659304941945443,"sd":0.05179374715674442,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5766851055684378,"sd":0.04582951427712403,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.503430183470071,"sd":0.04459793945177279,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.4030172558648468,"sd":0.027861274941953677,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.39162562562289294,"sd":0.05641587516544004,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.4552547434012159,"sd":0.022612113760275828,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.47620069973624185,"sd":0.02588416920899869,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.123348696439943,"sd":0.08423987325835493,"seeds":[1,3,4],"note":0,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3353173605987125,"sd":0.049931803870473544,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.44909534170685106,"sd":0.05596079562463969,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.26348779341849005,"sd":0.11868465717634435,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"orthrus-4track","mean":-0.14725146903883837,"sd":0.01653509452332437,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.5214138148511049,"sd":0.05430649237012042,"seeds":[1,2,3],"note":42,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.3047872387942585,"sd":0.07471914700313662,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnaernie","mean":0.5322904434423764,"sd":0.07894619054333178,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnafm","mean":0.5244632387891836,"sd":0.01951677692020746,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"splicebert","mean":0.5573288650207693,"sd":0.052097888338021876,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false}]},{"id":"mean_ribosome_loading","benchmark":"ours_regression","name":"Mean Ribosome Loading Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8482620251684855,"sd":0.006177985762629998,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"baseline_cnn","mean":0.7435677812314977,"sd":0.006678617775464983,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"baseline_lstm","mean":0.860261103567349,"sd":0.0012218313443001611,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"borzoi","mean":0.7784408672118542,"sd":0.0029609908260040505,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5633820551737895,"sd":0.03465763757354726,"seeds":[1,2,8],"note":10,"rank":41.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8479003413113833,"sd":0.027760968780155916,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.8511914328802087,"sd":0.0024379263841822554,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"dnabert-3","mean":0.8162468605138663,"sd":0.0018844228048689464,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-4","mean":0.8207044585886011,"sd":0.0035820525264496045,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert-5","mean":0.8193245741290912,"sd":0.004583207544098516,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert-6","mean":0.8253080876881285,"sd":0.004062226579455839,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert2","mean":0.771400935483034,"sd":0.0013479603734642082,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"enformer","mean":0.7413198592822053,"sd":0.0034285371117874284,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"evo2_1b_base","mean":0.821360516910159,"sd":0.006763783938217961,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7111305527268145,"sd":0.005118057515892893,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7850730794340611,"sd":0.008246414615754767,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7523311473940032,"sd":0.006901276258906733,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7329147785073448,"sd":0.007776985855782756,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7831827570591693,"sd":0.004346348665551905,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7911272947667903,"sd":0.006519587607357733,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7714769486387502,"sd":0.01650994722771123,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"phylogpn","mean":0.7698126767924941,"sd":0.010577351586765777,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"grover","mean":0.7823136303781587,"sd":0.002874861146214478,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_large_1m","mean":0.8160222424941818,"sd":0.003411981227994975,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.8116839330569966,"sd":0.006178069278588689,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5226461441818985,"sd":0.017127036372275757,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8466259781594397,"sd":0.00921859542166946,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.8478944215155292,"sd":0.001947382438520266,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.83910643007858,"sd":0.007589933888163231,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7818994561788891,"sd":0.001878278108088572,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7600868845835723,"sd":0.00265069193287445,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7358788466366116,"sd":0.0061171079990310125,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5961099589477215,"sd":0.011679859587928639,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7668020762965767,"sd":0.007709553671127639,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7987577017433481,"sd":0.011392821443990956,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.828867803383673,"sd":0.0017292201122573348,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.773285889376905,"sd":0.01763630513294815,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"orthrus-4track","mean":0.7841834537444702,"sd":0.0014592581054184799,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"rinalmo","mean":0.861862365541051,"sd":0.006537402219882687,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"rnabert","mean":0.2607775103998836,"sd":0.035213807599317276,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.8124264194868432,"sd":0.002610958642574497,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"rnafm","mean":0.8299219614262817,"sd":0.007486732724867598,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"splicebert","mean":0.8641326812615734,"sd":0.0007546114209449875,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true}]},{"id":"mean_ribosome_loading_human","benchmark":"ours_regression","name":"Mean Ribosome Loading Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.76174874050935,"sd":0.00282866557410696,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"baseline_cnn","mean":0.3469161003766386,"sd":0.03630192914104284,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.7573034959413977,"sd":0.0033348973737643488,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"borzoi","mean":0.6200465612877918,"sd":0.003252253911827445,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6095304669817482,"sd":0.02135388905415403,"seeds":[1,2,4],"note":0,"rank":32.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7643042384394911,"sd":0.008632922935847748,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.744622720532951,"sd":0.021233870851581065,"seeds":[1,3,5],"note":0,"rank":10.0,"top_score":false},{"model":"dnabert-3","mean":0.7430893765973187,"sd":0.005403407429680736,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-4","mean":0.7330196686458271,"sd":0.0020401989909823908,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert-5","mean":0.731231852330868,"sd":0.004521920368924129,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-6","mean":0.7419849739536457,"sd":0.0108540930892025,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert2","mean":0.6777764337336749,"sd":0.03858970979497558,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"enformer","mean":0.41536513486606563,"sd":0.023088772649563803,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7269793099882016,"sd":0.019545119807665177,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5971105752080156,"sd":0.009389746770622716,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6814004475229168,"sd":0.012548535889461997,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5112425669914947,"sd":0.03988017394115926,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5020054487448581,"sd":0.01335123719579621,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.70127647205595,"sd":0.007953080968350443,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6671394859980649,"sd":0.028555623838274747,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7288521428039187,"sd":0.007011147545439075,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"phylogpn","mean":0.2536544392241265,"sd":0.06577562815352661,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"grover","mean":0.7215961970629047,"sd":0.01185380254522301,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7523337786420325,"sd":0.012173464438566513,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7623703849647893,"sd":0.016392632348338227,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.36152051029482524,"sd":0.008022639708247736,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7621849152483804,"sd":0.005004302571976144,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7684154856327822,"sd":0.003906479443114582,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7402054904058298,"sd":0.010334542056870615,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6072121105642367,"sd":0.008112282701888268,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6812224204273704,"sd":0.007484104513722001,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5087153549394843,"sd":0.013420588792560069,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4033158583851703,"sd":0.02016406403595314,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6632507787950287,"sd":0.018686396423106072,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6828457976061388,"sd":0.004982233901179518,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7174349532510327,"sd":0.010864371882810316,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6175071191420408,"sd":0.0212435377916678,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"orthrus-4track","mean":0.6930217662747473,"sd":0.019776026367231258,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"rinalmo","mean":0.7775301902376514,"sd":0.013420793704338798,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"rnabert","mean":0.08613607727652071,"sd":0.061371767715652,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7460862697800216,"sd":0.004437889554653948,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnafm","mean":0.7372242395659558,"sd":0.008032758801194959,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"splicebert","mean":0.7110330909978543,"sd":0.002683785895755558,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false}]},{"id":"melting_point","benchmark":"ours_regression","name":"Melting Point Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.7020781329419047,"sd":0.006777042068618809,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"baseline_cnn","mean":0.5689711463797394,"sd":0.012816392358610632,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"baseline_lstm","mean":0.7088170133327967,"sd":0.006298153550254776,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"borzoi","mean":0.7056274573391167,"sd":0.006100992103879209,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.68581267417236,"sd":0.016053199200515605,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6899146765592197,"sd":0.023782237701604453,"seeds":[1,2,4],"note":0,"rank":16.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5338277975538978,"sd":0.019461127999008305,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-3","mean":0.35099133884951894,"sd":0.03794455583458277,"seeds":[3,4,6],"note":8,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.5232866713669134,"sd":0.024969925846913998,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-5","mean":0.6863901456821134,"sd":0.020937505084004562,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"dnabert-6","mean":0.6565599293424155,"sd":0.07521574823924428,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"dnabert2","mean":-0.04748519748634969,"sd":0.014463392849972959,"seeds":[1,2,3],"note":43,"rank":42.0,"top_score":false},{"model":"enformer","mean":0.7138804566056738,"sd":0.0026062608810875584,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6392008755789617,"sd":0.07937946500298526,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6993300723975726,"sd":0.005189349324832089,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6699798971996259,"sd":0.008384751069230123,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6950508277248035,"sd":0.012059659931122305,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7146226626692975,"sd":0.007030093938676004,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6574540237475889,"sd":0.009029024704184627,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5629149577363843,"sd":0.07528231630201952,"seeds":[3,5,7],"note":44,"rank":32.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7340518519144325,"sd":0.007373380343688943,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":-0.06582954510181523,"sd":0.03624921819770936,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"grover","mean":0.6119164736816979,"sd":0.06227155695709262,"seeds":[1,2,4],"note":15,"rank":27.0,"top_score":false},{"model":"hyena_large_1m","mean":0.695604055609994,"sd":0.0033614604504045064,"seeds":[6,7,8],"note":45,"rank":13.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.2592708120894444,"sd":0.042296256853304494,"seeds":[1,2,4],"note":6,"rank":37.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6894130751468398,"sd":0.020663436245453286,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7573424211387133,"sd":0.012838847242882057,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7771980305563487,"sd":0.012942905155505832,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.7552173314748779,"sd":0.008239356509227042,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.644931811648998,"sd":0.0019579055116516397,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.681202091337938,"sd":0.01987581222013574,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5835930815747491,"sd":0.032046485000305755,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5859852157010667,"sd":0.027469224993087068,"seeds":[1,2,3],"note":46,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7002612922004467,"sd":0.0034549599826709564,"seeds":[1,2,3],"note":46,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6467483490800001,"sd":0.055399236818272884,"seeds":[1,2,3],"note":46,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6999411948575885,"sd":0.010286559954561292,"seeds":[1,2,3],"note":46,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6952970362401112,"sd":0.004855852133031667,"seeds":[1,2,3],"note":46,"rank":14.0,"top_score":false},{"model":"orthrus-4track","mean":0.5853607401631288,"sd":0.0023579545521098225,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.07519205175917024,"sd":0.03140823019092717,"seeds":[4,5,11],"note":47,"rank":39.0,"top_score":false},{"model":"rnabert","mean":0.18100319971792125,"sd":0.0992735176535467,"seeds":[1,13,14],"note":48,"rank":38.0,"top_score":false},{"model":"rnaernie","mean":-0.01942901340464103,"sd":0.022740116637535654,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rnafm","mean":0.47093752174790393,"sd":0.004905595023363795,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"splicebert","mean":-0.017802907198900854,"sd":0.04918845566783796,"seeds":[1,2,6],"note":15,"rank":40.0,"top_score":false}]},{"id":"mrfp_expression","benchmark":"ours_regression","name":"mRFP Expression Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8022334533393809,"sd":0.02276820046232477,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"baseline_cnn","mean":0.5764274031156712,"sd":0.01703125086413144,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"baseline_lstm","mean":0.22007358593454956,"sd":0.023895539472650214,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"borzoi","mean":0.6645430360790027,"sd":0.041687085759897796,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.25407854996555385,"sd":0.044228471602374475,"seeds":[1,3,5],"note":49,"rank":38.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.24994142378173875,"sd":0.04788398716637098,"seeds":[1,2,3],"note":32,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.4870952697588768,"sd":0.15196839206864113,"seeds":[5,11,8],"note":50,"rank":34.0,"top_score":false},{"model":"dnabert-3","mean":0.77932472140323,"sd":0.0145383195519905,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-4","mean":0.7773631973542412,"sd":0.010878350597372256,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-5","mean":0.8012782784004692,"sd":0.0224206096649983,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-6","mean":0.8382744510815758,"sd":0.023692900708210115,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"dnabert2","mean":0.22476359410445426,"sd":0.13551462091910202,"seeds":[1,3,5],"note":51,"rank":40.0,"top_score":false},{"model":"enformer","mean":0.1732084175855526,"sd":0.06618722445236688,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5246871449220288,"sd":0.14355882345145365,"seeds":[1,4,5],"note":0,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6257140067377565,"sd":0.054047353677706036,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6746712991481835,"sd":0.06354398147308245,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5316966205044853,"sd":0.04422938373127883,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5107494421263339,"sd":0.05846172276362779,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6224833270125071,"sd":0.09639821305582283,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5971239727383207,"sd":0.016471069562750112,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8128568303824997,"sd":0.014278350232100388,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.5623878528948906,"sd":0.1067698356944702,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"grover","mean":0.6906727160795212,"sd":0.09023007885308668,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5319666781588,"sd":0.09228521914569635,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6535553682279897,"sd":0.01432159917202339,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6969957322046453,"sd":0.035189218009388325,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8039505832221313,"sd":0.027464857275244448,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7887305205406842,"sd":0.08850460111318112,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7358329087678949,"sd":0.03943969640299336,"seeds":[1,2,5],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7742760392091214,"sd":0.023268007815342592,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7684622919764185,"sd":0.01710540299812995,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8010495805635385,"sd":0.02659181771402286,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7264100474874029,"sd":0.018788759543917406,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6317626578824952,"sd":0.04306816701884264,"seeds":[1,2,3],"note":46,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5749320454035005,"sd":0.06249363821143815,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5972320786029476,"sd":0.05533758709078364,"seeds":[1,2,3],"note":46,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.37257726621140447,"sd":0.07019091639012057,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"orthrus-4track","mean":0.4865571288293302,"sd":0.04409272102649331,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.7831372331733463,"sd":0.03370328092136934,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.33173542367150033,"sd":0.00722661467518764,"seeds":[1,2,5],"note":52,"rank":37.0,"top_score":false},{"model":"rnaernie","mean":0.8029801295936959,"sd":0.08035143410259842,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnafm","mean":0.66019875572763,"sd":0.08045338832759458,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.12980563733841605,"sd":0.10948942447712093,"seeds":[4,6,7],"note":35,"rank":43.0,"top_score":false}]},{"id":"mrna_stability","benchmark":"ours_regression","name":"mRNA Stability Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.35341527934834877,"sd":0.004036504616943558,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"baseline_cnn","mean":0.20572593283787685,"sd":0.004701151742237076,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.2670763098432871,"sd":0.006291684682161522,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.2838067638269546,"sd":0.015075776774637286,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.3118348339621685,"sd":0.0006735749157043137,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3169614720686908,"sd":0.00919954280900379,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.2169581198229656,"sd":0.018620288844721705,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.23443914758428713,"sd":0.008877687054648096,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.24774709462274608,"sd":0.008902128569080079,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-5","mean":0.23100847972609947,"sd":0.016835447470710183,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-6","mean":0.23648139043542676,"sd":0.010642419915948366,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert2","mean":0.29118504016030133,"sd":0.005534040410090781,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"enformer","mean":0.2832963074355234,"sd":0.0034201754900303645,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"evo2_1b_base","mean":0.33924099674307234,"sd":0.02403055837029351,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.24603924605392372,"sd":0.004699143279024835,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.29770158509648803,"sd":0.008059175309545936,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.2699435370898738,"sd":0.026979014646560016,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.26894287641194303,"sd":0.03793363160392435,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3130079841770912,"sd":0.01292639386161519,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2924918714771249,"sd":0.003769079756955433,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.42528223732016385,"sd":0.011779114089620957,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.29471901762565644,"sd":0.0029227034878300806,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"grover","mean":0.27279779241224744,"sd":0.0098101933213696,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_large_1m","mean":0.27912114711483543,"sd":0.008802463964131079,"seeds":[12,14,15],"note":53,"rank":29.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.29551461614659513,"sd":0.011516631409079115,"seeds":[1,2,3],"note":54,"rank":21.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.23961806083683387,"sd":0.025790552190660634,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.3548186628275537,"sd":0.0037580317278605438,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.38424125984515217,"sd":0.013455898731773764,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3564522962068301,"sd":0.005548292327602952,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.31512128713841103,"sd":0.012962704311715548,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.37657858356575313,"sd":0.0053608635161562935,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.29806535713254084,"sd":0.00639352882949449,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3032031290481902,"sd":0.007392975230775094,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3593513125430201,"sd":0.0044942125094324685,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.36358440999890423,"sd":0.005631878274796319,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.36471036479572316,"sd":0.016678372516337817,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3292983427734625,"sd":0.006765968378114041,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"orthrus-4track","mean":0.2467573675728408,"sd":0.0017230246439633497,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.38529793060850037,"sd":0.005622808871360689,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"rnabert","mean":0.1598836827111825,"sd":0.011382819142563777,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.28584532807069335,"sd":0.01300676375066552,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rnafm","mean":0.29504151018795277,"sd":0.015501030988342088,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"splicebert","mean":0.33608543167409866,"sd":0.006714027687671888,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false}]},{"id":"muscle_el","benchmark":"ours_regression","name":"Expression Level Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.23009713381990057,"sd":0.050247498669376445,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"baseline_cnn","mean":0.14166125409676392,"sd":0.06201642304568054,"seeds":[1,2,4],"note":0,"rank":24.0,"top_score":false},{"model":"baseline_lstm","mean":0.21477609660690278,"sd":0.020019691907590717,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"borzoi","mean":0.159359572380487,"sd":0.06776796106778613,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.07719123586988191,"sd":0.09226425862549119,"seeds":[2,3,11],"note":35,"rank":36.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.08039716059254426,"sd":0.010024824109190547,"seeds":[1,2,11],"note":35,"rank":35.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.1322564389982465,"sd":0.06446266739952708,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert-3","mean":0.1251183404501413,"sd":0.0693650987590883,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.359530134630116,"sd":0.049568823469155886,"seeds":[1,2,4],"note":null,"rank":2.0,"top_score":false},{"model":"dnabert-5","mean":0.15044721868600822,"sd":0.08563224612126119,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-6","mean":0.12018264934520319,"sd":0.05667008480528657,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert2","mean":0.23939468360157548,"sd":0.0783099206705383,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"enformer","mean":0.22745691558974934,"sd":0.09512479925804569,"seeds":[2,3,4],"note":0,"rank":13.0,"top_score":false},{"model":"evo2_1b_base","mean":0.16847178554570397,"sd":0.07697480000464743,"seeds":[1,3,4],"note":0,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.0053878287354517315,"sd":0.08921678486117729,"seeds":[2,3,4],"note":0,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.03566610149560604,"sd":0.0688991194155326,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.14007152090121794,"sd":0.08933239460603676,"seeds":[1,2,4],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.0491765750964388,"sd":0.04717810206056541,"seeds":[1,2,4],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.035562712349148345,"sd":0.040744070752084506,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.10416592706682498,"sd":0.09314768356425027,"seeds":[2,3,5],"note":0,"rank":34.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.1291840635074969,"sd":0.061460544924518044,"seeds":[2,5,7],"note":null,"rank":30.0,"top_score":false},{"model":"phylogpn","mean":0.3200646777054502,"sd":0.05070222078001074,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"grover","mean":0.011222997304500784,"sd":0.09664471162602313,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"hyena_large_1m","mean":0.15133760310899466,"sd":0.05270177528817416,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.19890803795760345,"sd":0.09339106672954822,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2703325998393886,"sd":0.14667653725588825,"seeds":[3,4,5],"note":0,"rank":7.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.13882531488520253,"sd":0.07735682729668542,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.03279119622276156,"sd":0.0906956187941598,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.26071326933771183,"sd":0.09001012735460323,"seeds":[1,4,5],"note":0,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.35271209272711634,"sd":0.017178326544191675,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.25629546271145415,"sd":0.030241276172157234,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.1465693944730607,"sd":0.04274220002944424,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.29047795669064114,"sd":0.045010950655457196,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.349336708675323,"sd":0.06411736887515185,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.13800097456838772,"sd":0.11898102850542583,"seeds":[3,5,6],"note":0,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.2331311507566441,"sd":0.0999270303612109,"seeds":[1,2,4],"note":0,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":-0.11506322262615318,"sd":0.04269592585670685,"seeds":[3,5,6],"note":null,"rank":43.0,"top_score":false},{"model":"orthrus-4track","mean":0.19795357265360516,"sd":0.0725115064495197,"seeds":[2,3,4],"note":null,"rank":17.0,"top_score":false},{"model":"rinalmo","mean":0.12899749570296187,"sd":0.042422381775897695,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"rnabert","mean":0.16868231823517496,"sd":0.08581741084873569,"seeds":[1,13,16],"note":55,"rank":18.0,"top_score":false},{"model":"rnaernie","mean":0.20577743547346825,"sd":0.09912615098798942,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.13428561241297007,"sd":0.055086260340472336,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"splicebert","mean":0.3765474224175975,"sd":0.11698891667897703,"seeds":[1,3,5],"note":0,"rank":1.0,"top_score":true}]},{"id":"muscle_te","benchmark":"ours_regression","name":"Translation Efficiency Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":9.69207972153797e-05,"sd":0.020253917973398927,"seeds":[1,3,4],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_cnn","mean":0.15540110184660166,"sd":0.15739451659240533,"seeds":[1,3,4],"note":0,"rank":28.0,"top_score":false},{"model":"baseline_lstm","mean":0.014176837783181668,"sd":0.04915988950615415,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.24340046598398843,"sd":0.11254464596177238,"seeds":[2,3,4],"note":56,"rank":18.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.06587598111812562,"sd":0.06779309131961425,"seeds":[1,3,4],"note":0,"rank":34.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.20638594143636282,"sd":0.1568832823792078,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.24876353760622996,"sd":0.046557294824691405,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert-3","mean":0.28687234204625833,"sd":0.05914306537534334,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert-4","mean":0.3615446857442856,"sd":0.13347731218728584,"seeds":[3,4,5],"note":0,"rank":5.0,"top_score":false},{"model":"dnabert-5","mean":0.24266104087275098,"sd":0.06394574952934647,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"dnabert-6","mean":0.39575085949200794,"sd":0.10945747590236896,"seeds":[1,4,5],"note":0,"rank":3.0,"top_score":false},{"model":"dnabert2","mean":0.21750575769779498,"sd":0.1269881429781453,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"enformer","mean":0.2828433948333751,"sd":0.20904262155286654,"seeds":[1,2,10],"note":0,"rank":8.0,"top_score":false},{"model":"evo2_1b_base","mean":0.20136323326537403,"sd":0.132746732794227,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4069925237510828,"sd":0.035230992074315085,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.25883942926206216,"sd":0.08404851364288246,"seeds":[2,3,4],"note":0,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.14828378725290556,"sd":0.10123701954079968,"seeds":[2,3,5],"note":0,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.2441761888387822,"sd":0.10792823822677446,"seeds":[1,3,5],"note":0,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3112088734382774,"sd":0.03321111552503245,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.17645333069356864,"sd":0.0472313228328798,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.23768654209753404,"sd":0.08668564461036311,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"phylogpn","mean":-0.1888667335722703,"sd":0.04347958799237151,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"grover","mean":0.11976958326521812,"sd":0.009965725204910608,"seeds":[1,5,6],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_large_1m","mean":0.24797407291413176,"sd":0.14287628639985842,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.04149182304257212,"sd":0.09818194663393313,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.17997800309104184,"sd":0.19526059274814106,"seeds":[2,3,4],"note":0,"rank":26.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.09273952446858992,"sd":0.035062364593519676,"seeds":[2,4,7],"note":null,"rank":31.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.2778861925591876,"sd":0.1261121510039244,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.0018482092801849436,"sd":0.11174141864635734,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.27688405842330416,"sd":0.0490550137249359,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.24427401707910001,"sd":0.07649172473998422,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.3831732499548921,"sd":0.030989792737642456,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4100577533659637,"sd":0.05731808921556672,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.0740355383818914,"sd":0.08072640046234988,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":-0.049383395374698234,"sd":0.008323237088384042,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.2702575461673017,"sd":0.0452760928724952,"seeds":[1,4,5],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.24265584895639536,"sd":0.05708013912179443,"seeds":[1,4,5],"note":0,"rank":20.0,"top_score":false},{"model":"orthrus-4track","mean":0.18200416420517673,"sd":0.1346434448119208,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"rinalmo","mean":0.020851382133132017,"sd":0.14599905985747233,"seeds":[1,4,5],"note":0,"rank":38.0,"top_score":false},{"model":"rnabert","mean":0.06848332780158062,"sd":0.11179467064339209,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rnaernie","mean":0.04227065270216861,"sd":0.155010134022943,"seeds":[1,2,5],"note":35,"rank":36.0,"top_score":false},{"model":"rnafm","mean":0.2651699937846461,"sd":0.10531970322836041,"seeds":[1,3,5],"note":0,"rank":12.0,"top_score":false},{"model":"splicebert","mean":0.053103696804653086,"sd":0.07187885074793372,"seeds":[2,3,5],"note":8,"rank":35.0,"top_score":false}]},{"id":"pa_athaliana","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6032114648037185,"sd":0.011330150617386075,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.4130884472055641,"sd":0.032550536965528,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.5523987088361102,"sd":0.014049021562007012,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"borzoi","mean":0.5595312103042872,"sd":0.006365955235447072,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5314593407022263,"sd":0.054878966189646355,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5114373069532131,"sd":0.09359185907541671,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5643165646175081,"sd":0.011284648474692942,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-3","mean":0.4709284814004376,"sd":0.015709146926188817,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.4642504648876747,"sd":0.005347044004389388,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-5","mean":0.4837196543256707,"sd":0.013338120977572983,"seeds":[1,3,4],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.44051254680732393,"sd":0.0278309653012977,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert2","mean":0.5466300519413546,"sd":0.0190560484081425,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"enformer","mean":0.445516223375356,"sd":0.01901270202120251,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6208091800658189,"sd":0.01598004579331922,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.49320276988982775,"sd":0.0034481532178793767,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5176925908857298,"sd":0.024376670891807548,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.46922521699532566,"sd":0.08229021951505026,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.4877586840362127,"sd":0.015688910916784402,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.550357131801082,"sd":0.006135921286596365,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5065845196434187,"sd":0.008616811490349693,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6901101861034826,"sd":0.011365531557971424,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.561591965577322,"sd":0.010931437493517167,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"grover","mean":0.5268495623387648,"sd":0.000623084224347573,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"hyena_large_1m","mean":0.444348143751472,"sd":0.08246377405364132,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4966887758186315,"sd":0.019117279305849454,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5238758248858824,"sd":0.027752322777210736,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6409868152740475,"sd":0.010054409459996229,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6871638639564145,"sd":0.007515957963275503,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6164190931578317,"sd":0.01832931131449765,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6202903316613048,"sd":0.004067046118923934,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6398896132162677,"sd":0.0073896764978686235,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6027145955394025,"sd":0.015345310382384222,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5996108967815305,"sd":0.005946938408478903,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6282514916094096,"sd":0.01640221128524321,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6263719897600529,"sd":0.003689492737356176,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6452639101171519,"sd":0.006549659674818388,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6246712627547005,"sd":0.003757404922209707,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"orthrus-4track","mean":0.48244253567993883,"sd":0.009232703715149986,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.6688735140495773,"sd":0.0021329223495546524,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"rnabert","mean":0.35008901196412073,"sd":0.01850449370675524,"seeds":[1,2,4],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.5404857588502164,"sd":0.009820402914938223,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.5546267950352585,"sd":0.014646966512686379,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"splicebert","mean":0.6001541301589689,"sd":0.001654344933474185,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false}]},{"id":"pa_dmelanogaster","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.597203940492277,"sd":0.0007887396776243121,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"baseline_cnn","mean":0.4144531813119883,"sd":0.006219238472689612,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.5595625382461725,"sd":0.005309152684077652,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"borzoi","mean":0.5645200745155329,"sd":0.010154464491377184,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5680180541280325,"sd":0.019871943457733496,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5901890775858737,"sd":0.010301428525537195,"seeds":[1,3,4],"note":0,"rank":16.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5089293508367901,"sd":0.01854703154260997,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.5489694113565814,"sd":0.018370413662926178,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"dnabert-4","mean":0.531432295046116,"sd":0.012958061509393569,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.5480336843241883,"sd":0.005941548510524552,"seeds":[1,3,5],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-6","mean":0.5155031411657233,"sd":0.009438179037448301,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.5047558853339086,"sd":0.1143618401949473,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.5098823064228376,"sd":0.025388736095110602,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5713141920728122,"sd":0.04646723349400684,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5151759510032861,"sd":0.014589986200794266,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.558875399724558,"sd":0.020884165897554076,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5310160296876897,"sd":0.014943157211040213,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5384403033297182,"sd":0.013781282192223746,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.588480437450408,"sd":0.004701489376285707,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.565566744084836,"sd":0.014067720263602743,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6876134573922873,"sd":0.01087734629386652,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.5472834895093847,"sd":0.008099818845019549,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"grover","mean":0.5503393100243467,"sd":0.0188162457247437,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5243803853302815,"sd":0.023493346636452507,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5285468843722617,"sd":0.013475549445501663,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5437332534343446,"sd":0.002414276711103908,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6544639938235643,"sd":0.006418124320532709,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7126395069135919,"sd":0.010609378157440525,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.6571695589705322,"sd":0.004654980433313783,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6484088295039577,"sd":0.005130015825273235,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6789358987018042,"sd":0.002453388495107315,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6115307523214029,"sd":0.009149222793436213,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6164771172940817,"sd":0.003458055697525942,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.644537259017188,"sd":0.017285607957564623,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6516902507149278,"sd":0.006734578048966348,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6590312028816944,"sd":0.010920427466351504,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6254239295040623,"sd":0.0077627564320552145,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"orthrus-4track","mean":0.50498685877889,"sd":0.03842049828684316,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rinalmo","mean":0.6658851283355601,"sd":0.004718476336945094,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.40694431614815013,"sd":0.01003416591584738,"seeds":[1,2,4],"note":6,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.5710806096296738,"sd":0.01529391053111464,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"rnafm","mean":0.5603394054397812,"sd":0.013425069228407955,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.6391693755868558,"sd":0.006852738456367848,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false}]},{"id":"pa_ecoli","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6478973683387722,"sd":0.005362182287617756,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.512497919156923,"sd":0.01732108526534518,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.5871105043363434,"sd":0.005283005769659885,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"borzoi","mean":0.5513470374146989,"sd":0.001969405685789194,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6025532716762511,"sd":0.053905183048473644,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6045732116087144,"sd":0.020327975167298673,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5408765083622532,"sd":0.04528738038939797,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.5650090320858692,"sd":0.06323153772154536,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.5440603607706355,"sd":0.02744660322473994,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.5928991614670407,"sd":0.01331473256547985,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-6","mean":0.5614757414784806,"sd":0.01943109195193145,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.622435664754951,"sd":0.006943639150380663,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"enformer","mean":0.5126716060035341,"sd":0.02238018438118485,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6602483788247846,"sd":0.013701602648089041,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5670399722656326,"sd":0.021800159458242773,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5980007808911069,"sd":0.008062343775130983,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6021114756363247,"sd":0.021205252134065634,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5842092783165117,"sd":0.03166828542298318,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.610791830499959,"sd":0.01762938589818468,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5983507941336882,"sd":0.01759510226852458,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6884366362089298,"sd":0.01541439574594008,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"phylogpn","mean":0.594078587954114,"sd":0.02470455478688238,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"grover","mean":0.6180151553090355,"sd":0.009674435542768508,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5685904800683957,"sd":0.03267236276273847,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.558004332680038,"sd":0.009140555601793915,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5771463633826589,"sd":0.03141583115820422,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6673712189933969,"sd":0.013006054895698506,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7196807489696709,"sd":0.02205327991648778,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.6784618527011909,"sd":0.003958662379405071,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6546787596307512,"sd":0.00647839654675706,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6943561306466529,"sd":0.010522903218277596,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6147825587179012,"sd":0.02691105544676342,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6375821434003784,"sd":0.007891141994931414,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6753164811439873,"sd":0.006979758216430334,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6945680444752057,"sd":0.003723729373285892,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6899548532545755,"sd":0.0014320950233154418,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6625447734042014,"sd":0.007627401551467501,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.574344616589033,"sd":0.012110635902348035,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.6934456860759418,"sd":0.01649174623683717,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.41693262499867456,"sd":0.05753788141020057,"seeds":[1,2,5],"note":0,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.6354331030563968,"sd":0.017933081686592935,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"rnafm","mean":0.6168290079970635,"sd":0.021138939342498518,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"splicebert","mean":0.679921179055318,"sd":0.005673050015477283,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false}]},{"id":"pa_hsapiens","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.4765122024765602,"sd":0.00448426776186984,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.2140881039955525,"sd":0.017876712139362372,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"baseline_lstm","mean":0.3732785959687652,"sd":0.012163728967083204,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"borzoi","mean":0.4010407679802605,"sd":0.015738845618828406,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.3728727080269638,"sd":0.0634082372370007,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3460405804882956,"sd":0.04886875055326075,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.34471622954253905,"sd":0.022710719687075786,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.3518657696526783,"sd":0.037373770060066945,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.35234895832373825,"sd":0.012417955586238661,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-5","mean":0.3490842848384101,"sd":0.012108073886548252,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-6","mean":0.32810598289900184,"sd":0.0002849436461832421,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert2","mean":0.4193448710867013,"sd":0.022426355146565612,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"enformer","mean":0.37221400583252745,"sd":0.01738767604987798,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"evo2_1b_base","mean":0.4819537206644772,"sd":0.01642601919153826,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.356056164357713,"sd":0.002462114668985845,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.38301241110804574,"sd":0.019751631296309694,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.36777206443702326,"sd":0.015076162683483229,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.3624472467798228,"sd":0.007991420608042643,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.41758566600632935,"sd":0.01035009064626978,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3843969457500651,"sd":0.014085500074336258,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6204373184633593,"sd":0.007224996608378493,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.4761457492577659,"sd":0.0114763081608208,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"grover","mean":0.39203388545025836,"sd":0.0073129095595432176,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3976993867535545,"sd":0.00768490411830002,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3849893585207174,"sd":0.005739862088153668,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.39994800356470267,"sd":0.015537450925287867,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5316810507924327,"sd":0.017981543681354444,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5993452720844957,"sd":0.016708367209204853,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5061328745913396,"sd":0.03111650729542897,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.49045694207362517,"sd":0.004988498597107666,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5386274660219001,"sd":0.013373683553906926,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.45683803798073414,"sd":0.028054649236474265,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.477325117046073,"sd":0.006043966732084794,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.4833039049241772,"sd":0.008943883809184362,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5053525353481407,"sd":0.012071344769227003,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5354914091060032,"sd":0.019873381088934328,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.4726417558992075,"sd":0.0031427938207327245,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"orthrus-4track","mean":0.374052557904728,"sd":0.017172414524838715,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.5536161818650789,"sd":0.008432394740642364,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"rnabert","mean":0.2721807106845641,"sd":0.016831190756271847,"seeds":[1,2,6],"note":6,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.41977422168248424,"sd":0.013739733711581837,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"rnafm","mean":0.3872599804578544,"sd":0.02386444803285896,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"splicebert","mean":0.48801773841505125,"sd":0.0036664155636061046,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false}]},{"id":"pa_scerevisiae","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6398650017048537,"sd":0.025814702295655953,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.472127729943968,"sd":0.017674279880797672,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.6264091026764502,"sd":0.006648151806340037,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"borzoi","mean":0.5924181459383369,"sd":0.01936090058443442,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.579208343452683,"sd":0.034409501807475665,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6326903043429032,"sd":0.005895956224148605,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6030306592068367,"sd":0.010450588699443436,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-3","mean":0.5288650729383865,"sd":0.022909461381855428,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.4810446760613534,"sd":0.03263411870304332,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-5","mean":0.48560688092992405,"sd":0.023905747032212436,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-6","mean":0.5016706361510602,"sd":0.018997499025584282,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert2","mean":0.5971311413687418,"sd":0.014865654203105951,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"enformer","mean":0.4882631381546869,"sd":0.02874805364633152,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6831318144698316,"sd":0.003711293570151269,"seeds":[1,3,4],"note":0,"rank":5.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5968402622682369,"sd":0.017531592418368446,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6034402673057134,"sd":0.0007901129466173781,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5697864455688454,"sd":0.020806554283891696,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5681317821267847,"sd":0.014864706234603056,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5892036789660602,"sd":0.016543713311396382,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5976111699888674,"sd":0.012219272713207144,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7255977752634575,"sd":0.0010519755195973449,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.6015676437492975,"sd":0.01571858107622025,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"grover","mean":0.5863306358703434,"sd":0.005889523468249901,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5695981485216516,"sd":0.008144385787292268,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5733951371535385,"sd":0.002447328466678863,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6169794708117691,"sd":0.021486785120234927,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6605426405429542,"sd":0.018793229572041534,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6898198182775639,"sd":0.01695078427527471,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6822236916686872,"sd":0.004039676427123487,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6806950686128701,"sd":0.008411734062041357,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6900015791309531,"sd":0.00107968149179233,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6365475748963828,"sd":0.007674473670853747,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6377605334570305,"sd":0.007177741765688833,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6769003328271749,"sd":0.01379420863997397,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6575642149411117,"sd":0.008894311823701376,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6780026259080549,"sd":0.007235293439855657,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6489633207048491,"sd":0.0108631349469666,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"orthrus-4track","mean":0.5806277928594215,"sd":0.029687402492442568,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"rinalmo","mean":0.6843020590872332,"sd":0.015149049463922514,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.21711262636993758,"sd":0.11688423024476126,"seeds":[1,2,4],"note":20,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.6241464938507612,"sd":0.016187543462201824,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"rnafm","mean":0.5967478618831197,"sd":0.0087794592713988,"seeds":[1,2,4],"note":0,"rank":27.0,"top_score":false},{"model":"splicebert","mean":0.6613937404446605,"sd":0.004629905186650123,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false}]},{"id":"pc3_el","benchmark":"ours_regression","name":"Expression Level Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.1977740067300407,"sd":0.009442316078111511,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.07078656310604882,"sd":0.01585820666429981,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"baseline_lstm","mean":0.11269606824920181,"sd":0.015136723799219176,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"borzoi","mean":0.08892515529830744,"sd":0.05468758185387548,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.09593783652378646,"sd":0.06670060164943699,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.15333511781867867,"sd":0.02393122617093603,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.07432496425703321,"sd":0.019184185038856688,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-3","mean":0.09073826055318596,"sd":0.031878748724820896,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-4","mean":0.09807350872293177,"sd":0.026085070300104714,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert-5","mean":0.09370408955698696,"sd":0.014266568453580044,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-6","mean":0.004509336142867973,"sd":0.08036908526739871,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert2","mean":0.07915739726783926,"sd":0.04169569277777469,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.1472990737286274,"sd":0.036922809310531775,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"evo2_1b_base","mean":0.11463060112840243,"sd":0.045237968920584556,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.07384556912951475,"sd":0.07635488198524547,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.13635881201144642,"sd":0.01646907216441056,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.1248188624033135,"sd":0.036262323608169555,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.11018286178176075,"sd":0.04545624260369491,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.15615945043395635,"sd":0.017661836658306108,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.12689362221807812,"sd":0.03241808594896187,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.10956323013317316,"sd":0.004674633208556404,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"phylogpn","mean":0.09462075768589544,"sd":0.023428946640690715,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.16725620351834988,"sd":0.01682688671318952,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"hyena_large_1m","mean":0.09467902480091768,"sd":0.04153043782857596,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.16078968330740032,"sd":0.016724138892646137,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.012996800465861063,"sd":0.06504389020239898,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.11576036211225887,"sd":0.00205069405868407,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.14060263905709092,"sd":0.02135203027769491,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.10401519629550746,"sd":0.0145399359797435,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.07508710282478828,"sd":0.016692922887402904,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.09581236665097187,"sd":0.022609740556262473,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.12135888375365839,"sd":0.01726113279071512,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.10928828078309462,"sd":0.015431889729040861,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.09160438313315288,"sd":0.046600164780154,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.12053560884212804,"sd":0.0020260317306001756,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.09687110019214099,"sd":0.02300906676279626,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.10718776566898029,"sd":0.018248345597399785,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.14447754881963112,"sd":0.014941024908224325,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rinalmo","mean":0.10290279807135295,"sd":0.00859639109399687,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rnabert","mean":0.007214110774933048,"sd":0.03295540485636205,"seeds":[1,2,6],"note":57,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.015283262803893456,"sd":0.057648666979233223,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnafm","mean":0.12194584935912373,"sd":0.004912249458086708,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"splicebert","mean":0.11573942970877887,"sd":0.028547795739999274,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false}]},{"id":"pc3_te","benchmark":"ours_regression","name":"Translation Efficiency Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.33052907868518283,"sd":0.010041823913898061,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.2059197680853621,"sd":0.05339329684267816,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.3461719698688009,"sd":0.03216831281792229,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"borzoi","mean":0.3476418648771002,"sd":0.008711291621082124,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.3575869256103251,"sd":0.010318256556790426,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3404972827921284,"sd":0.03179681351426947,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.22358422866204553,"sd":0.02082541630473126,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.3422405068415153,"sd":0.04725702361929441,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"dnabert-4","mean":0.3213733279703106,"sd":0.04678245820923144,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-5","mean":0.3638318987055152,"sd":0.007584913911838626,"seeds":[1,3,5],"note":null,"rank":1.0,"top_score":true},{"model":"dnabert-6","mean":0.35906181942812887,"sd":0.027136421409971394,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"dnabert2","mean":0.20172860457862246,"sd":0.09013206888203763,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"enformer","mean":0.31727486116691106,"sd":0.031077226610377,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3411596276643713,"sd":0.025836578357429054,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.247386570776945,"sd":0.0043769937662208975,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.34551876190828296,"sd":0.013943318756827737,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.205043911972379,"sd":0.054309496753703175,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.25938948073483264,"sd":0.02958385318376431,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3510476540418135,"sd":0.02757822533880848,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2645723653561514,"sd":0.05972100815298572,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.30379360970248,"sd":0.035664602831131084,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"phylogpn","mean":0.2565522011397976,"sd":0.06871303950488927,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"grover","mean":0.30798793661932417,"sd":0.012492322978332042,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.33196911268095464,"sd":0.0204894557711643,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3381716581120416,"sd":0.06760014916605098,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.15150747266196438,"sd":0.0449969628233273,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.3210508251204969,"sd":0.008935993173471081,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.325568870831375,"sd":0.017800711800944405,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3094665741762059,"sd":0.02154500841050638,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2695537712806035,"sd":0.056384669292398534,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.272350264252199,"sd":0.003556755427893412,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.27793836358887897,"sd":0.028463176133572063,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.27585219298633684,"sd":0.0038505568393291463,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.25890827260958205,"sd":0.007086362845269086,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.2664148094204596,"sd":0.030522967636188905,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.2760859816725834,"sd":0.024284765629150676,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.27883077628204267,"sd":0.007676975434120219,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"orthrus-4track","mean":0.2466256263992858,"sd":0.09109308300937172,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rinalmo","mean":0.30174377003799846,"sd":0.03831691808730234,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rnabert","mean":0.24458459099426272,"sd":0.01311872224102505,"seeds":[1,2,4],"note":0,"rank":38.0,"top_score":false},{"model":"rnaernie","mean":0.2677104931845163,"sd":0.02085568586443344,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rnafm","mean":0.32850595723139325,"sd":0.016413947251425004,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"splicebert","mean":0.2933933472967811,"sd":0.05474884200277352,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false}]},{"id":"programmable_rna_switches","benchmark":"ours_regression","name":"Programmable RNA Switches Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.7410002367882544,"sd":0.007893556726182622,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"baseline_cnn","mean":0.6607832899776243,"sd":0.006671295071327855,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.7346015493952286,"sd":0.002416073612370239,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"borzoi","mean":0.7185963396103502,"sd":0.002491978973469253,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7444621984191822,"sd":0.0028071191979900764,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7248948540627788,"sd":0.003908822804980017,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.7384654179284,"sd":0.0019040294339746067,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert-3","mean":0.7433983707810943,"sd":0.006575147875686496,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"dnabert-4","mean":0.73251464461012,"sd":0.020426578417445024,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-5","mean":0.741573644940797,"sd":0.0019967159280582895,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"dnabert-6","mean":0.7390491381917969,"sd":0.0008421528109815603,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert2","mean":0.7107355556841496,"sd":0.0028029825707426674,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"enformer","mean":0.6678927949379952,"sd":0.0038699987068310684,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7364662253177886,"sd":0.002447546383994848,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6442238948854467,"sd":0.0032305436447388858,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6982372616526862,"sd":0.0025612666583736507,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6867676130467535,"sd":0.003142239054667507,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.664715102842247,"sd":0.0164493613923332,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7106396549843612,"sd":0.0028859609722127833,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7054004162572832,"sd":0.0042188726968730396,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7349798871224291,"sd":0.002634323294846352,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"phylogpn","mean":0.6939305482112232,"sd":0.001072126184015048,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"grover","mean":0.6793430670530795,"sd":0.005904285034764873,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7268393283928848,"sd":0.007315779492559434,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7336679254481441,"sd":0.0012963241835623855,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6735623978726816,"sd":0.0026527337939667023,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.73940910455987,"sd":0.0014765377948573288,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7465800964170369,"sd":0.003755670546749371,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7359569785219623,"sd":0.0015534846494311347,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7391395005643036,"sd":0.0003076771679320581,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7482746331720483,"sd":0.00040967874932636264,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7232266964085147,"sd":0.0034964498863761352,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7162393252820745,"sd":0.0016023949915671763,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.725882401309715,"sd":0.0014939993302597564,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7327851840719897,"sd":0.0017220628797416733,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7338694732990799,"sd":0.004906004546480364,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7269775813260669,"sd":0.0020660268556417683,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"orthrus-4track","mean":0.6547791206517154,"sd":0.0011577337901668226,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.7460956879088647,"sd":0.004804466583329366,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"rnabert","mean":0.719979804555798,"sd":0.0019374901315234064,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rnaernie","mean":0.7509641398685173,"sd":0.0030590887198452513,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"rnafm","mean":0.744266732284482,"sd":0.00202132261019658,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"splicebert","mean":0.7439881114956288,"sd":0.003780119794196852,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false}]},{"id":"regulatory_activity_gm12878","benchmark":"ours_regression","name":"Regulatory Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8761387953598914,"sd":0.0023207988637795646,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.5994886737310539,"sd":0.0027977235725698607,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.6217518915787972,"sd":0.02445829283064284,"seeds":[1,2,3],"note":58,"rank":31.0,"top_score":false},{"model":"borzoi","mean":0.8553497266058222,"sd":0.0016272779005680033,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.797823079848774,"sd":0.004158710441557255,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7912876318804958,"sd":0.0035686444791153344,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.632076984358448,"sd":0.006832864345809132,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-3","mean":0.301587574570355,"sd":0.006848775270422053,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.30070897677489467,"sd":0.0007930217432365185,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.2941788215151032,"sd":0.004039030109012249,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-6","mean":0.2984981739157392,"sd":0.0033124853216900526,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert2","mean":0.7218350642008823,"sd":0.006463432924661927,"seeds":[1,3,4],"note":null,"rank":25.0,"top_score":false},{"model":"enformer","mean":0.8522642933440346,"sd":0.002536953903175118,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7853969679136817,"sd":0.0063869147631377965,"seeds":[1,2,3],"note":59,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.714265953387841,"sd":0.0018989288902603867,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7455596786843748,"sd":0.0024101616337694424,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7066875252714969,"sd":0.06427053652802894,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7393751217561925,"sd":0.003850504369042266,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7760438959469119,"sd":0.0013424191246241468,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7548704219199555,"sd":0.008872227854193606,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8031443865906351,"sd":0.003256464583694885,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.6127020001694863,"sd":0.025916303402163464,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.7418727505655135,"sd":0.004002394182656529,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6190863122471614,"sd":0.15289513957770726,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7355166581080895,"sd":0.001534103151433743,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5841549025026933,"sd":0.007329835553669256,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7831113431435955,"sd":0.0035484255171468466,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.800752731810045,"sd":0.002854156642497671,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.765405578010979,"sd":0.0030052002359452572,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7705408565857232,"sd":0.0011750639459766428,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7973894034853745,"sd":0.0010804027477347518,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7196525388440793,"sd":0.004523386247148907,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7030532154990361,"sd":0.0004927973048417976,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7677132587976416,"sd":0.0031947549563736996,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7881873165757789,"sd":0.002059309450180642,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.801052354766594,"sd":1.932789884285822e-05,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7384414699775617,"sd":0.002842879478303383,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"orthrus-4track","mean":0.5562683793792503,"sd":0.041013429929361474,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rinalmo","mean":0.7939168600357167,"sd":0.0025441113484648025,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.0525265114241918,"sd":0.04110027170864234,"seeds":[1,3,4],"note":60,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7703886467700056,"sd":0.0036367682688045917,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"rnafm","mean":0.5487989737423934,"sd":0.005749113354907833,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"splicebert","mean":0.5625658198081681,"sd":0.0010775652613092925,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false}]},{"id":"regulatory_activity_h1esc","benchmark":"ours_regression","name":"Regulatory Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.884681335571199,"sd":0.00045755286205862133,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6683520962598154,"sd":0.0033817651184222613,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"baseline_lstm","mean":0.7598848539748444,"sd":0.019057395261906744,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"borzoi","mean":0.8697267242140706,"sd":0.0009434501421268761,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8242656082181521,"sd":0.0023139273025358374,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8193425776772333,"sd":0.0029496674813537615,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.7376448214306971,"sd":0.003766518145321237,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-3","mean":0.36834701825217037,"sd":0.0050528305885532004,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.36573503966405846,"sd":0.0008078342304997973,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.35258064494919356,"sd":0.005281424619027003,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-6","mean":0.3588832496160714,"sd":0.0016949730661657571,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert2","mean":0.7702469791635134,"sd":0.00216231802750572,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"enformer","mean":0.8677132234813629,"sd":0.0013673974021302668,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8113324090503125,"sd":0.004874203519101859,"seeds":[1,2,3],"note":59,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7597283206304452,"sd":0.0022189137801039223,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.784349539901018,"sd":0.004985749082985029,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.709781585980981,"sd":0.050332104308801716,"seeds":[1,3,4],"note":61,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7682273998092851,"sd":0.0039289006986948935,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7982666288344705,"sd":0.0016485457623490996,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7903959504921179,"sd":0.0031882994988438286,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8149335024954627,"sd":0.0016240159400599087,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"phylogpn","mean":0.7292771528806823,"sd":0.0032413858408314035,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"grover","mean":0.7782129900621284,"sd":0.00040581418801419796,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7679707692141695,"sd":0.015381760570598813,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7645782778554019,"sd":0.0018060462171275416,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.655497106262941,"sd":0.0021906121153744866,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8097796764596914,"sd":0.0038664193544591563,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.822171125708903,"sd":0.004302742874218346,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.8050976318280607,"sd":0.0018962995721133358,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7945692233233739,"sd":0.0008172264521086778,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8130367046803163,"sd":0.0008182289308193709,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7559096340334742,"sd":0.0012783107605467607,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7462072708223728,"sd":0.0007026615715847489,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7910919570600117,"sd":0.002861065560199926,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.807162573543958,"sd":0.0008915251653620343,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8144878209221856,"sd":0.0072048580707065,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7737978820085644,"sd":0.0014236397855344096,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.7290701402964591,"sd":0.0023936832381896366,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rinalmo","mean":0.8204892227674777,"sd":0.0013284247831395202,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.1705072185699091,"sd":0.008542694352262317,"seeds":[1,2,4],"note":6,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.8022219897841981,"sd":0.0016397194597336582,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.6072703861598786,"sd":0.0036508220554179833,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"splicebert","mean":0.6197718242293941,"sd":0.0036074880090171076,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false}]},{"id":"regulatory_activity_hepg2","benchmark":"ours_regression","name":"Regulatory Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8391790400191504,"sd":0.003920686121359632,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.5706838069217819,"sd":0.0019206263223908017,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"baseline_lstm","mean":0.6709455050966107,"sd":0.04407368915496222,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"borzoi","mean":0.8271364986445452,"sd":0.0027329303604195607,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.775732850342306,"sd":0.004096365000532234,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.768135235909217,"sd":0.0016732131208561135,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.5945220156438823,"sd":0.0027918949394115356,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-3","mean":0.28328763685747166,"sd":0.00171845820629336,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.28012601166270773,"sd":0.004909059437964684,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.27671470890742067,"sd":0.003678739559337962,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-6","mean":0.2753723711169713,"sd":0.004897887953058277,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert2","mean":0.681553929545682,"sd":0.005716207602048007,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"enformer","mean":0.8162633189949916,"sd":0.004291053177109137,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7556926177472267,"sd":0.014530011748090788,"seeds":[1,2,3],"note":59,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6644207738573162,"sd":0.016308817510422874,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7200001029010928,"sd":0.009541011621386257,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7044482494528651,"sd":0.01632583264206583,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7088749089210812,"sd":0.004580202783395718,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7430137497749457,"sd":0.0030929622501966185,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7145810312464678,"sd":0.006976530090650457,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7671491869412469,"sd":0.007332453887501959,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"phylogpn","mean":0.572741254263244,"sd":0.011370653336700816,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.714861620280287,"sd":0.004819234249801342,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7012983133629581,"sd":0.03912518915396109,"seeds":[1,2,3],"note":62,"rank":25.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.728568233366053,"sd":0.01016324832901354,"seeds":[1,2,3],"note":12,"rank":19.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5487726667728512,"sd":0.002476295806802646,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7456593853603177,"sd":0.00347553733929592,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7750022049028964,"sd":0.01336565749428892,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7484507712040065,"sd":0.0038158082158477992,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7438374150820152,"sd":0.0017767048769436813,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7631402460934619,"sd":0.0025461115748468223,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6886944590106273,"sd":0.006241351667578536,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6629371260775752,"sd":0.013622089136904773,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7385923183525036,"sd":0.0002133485499837692,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7588240085861433,"sd":0.004549783521910031,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7666987817690072,"sd":0.01116618648180042,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6986104833151655,"sd":0.023814215750736063,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"orthrus-4track","mean":0.49434163058526803,"sd":0.044136927678645625,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rinalmo","mean":0.7632102555673039,"sd":0.018269071422374824,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"rnabert","mean":0.12340354519836176,"sd":0.001493061389055878,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7531003359814692,"sd":0.0053103741076382105,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"rnafm","mean":0.49419539833686743,"sd":0.0015537924847833344,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"splicebert","mean":0.513624488966475,"sd":0.0041418136902253615,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false}]},{"id":"regulatory_activity_imr90","benchmark":"ours_regression","name":"Regulatory Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9183968142287986,"sd":0.002999147788840054,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6702204668621657,"sd":0.0032989270335836594,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"baseline_lstm","mean":0.8116608789179897,"sd":0.010063526683214467,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"borzoi","mean":0.9024426573653943,"sd":0.0010547014879819744,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8543252508034676,"sd":0.0032849730426357453,"seeds":[1,2,5],"note":10,"rank":7.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8619276892794598,"sd":0.003349728782700235,"seeds":[1,7,8],"note":63,"rank":4.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.7237126498176183,"sd":0.01105324595200819,"seeds":[1,3,5],"note":64,"rank":33.0,"top_score":false},{"model":"dnabert-3","mean":0.28464482947380726,"sd":0.004822655598950889,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.28184885021105344,"sd":0.001053977519509682,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-5","mean":0.271048824143838,"sd":0.006848466733171528,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-6","mean":0.28403654882413454,"sd":0.0021559141145760883,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert2","mean":0.7643792592740805,"sd":0.03689293389833688,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"enformer","mean":0.8929912527514698,"sd":0.005081433288900054,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8446215626600742,"sd":0.0005919038383862222,"seeds":[1,2,3],"note":59,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7801279578413425,"sd":0.0010481923169474638,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8151760424627454,"sd":0.0009254901577459623,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.8068745484551033,"sd":0.005615150924195108,"seeds":[3,4,5],"note":0,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.8036799284833324,"sd":0.0015015091858097823,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.8307755831385902,"sd":0.003786266912508931,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7987344204230533,"sd":0.0071442374124635905,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8436065062406314,"sd":0.024446856849471718,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"phylogpn","mean":0.7080932660212353,"sd":0.00813020338281265,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"grover","mean":0.8117356231432425,"sd":0.0014301868060844012,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.8031755802280756,"sd":0.00562397813212224,"seeds":[1,2,3],"note":65,"rank":25.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7372855987514548,"sd":0.050520467420516794,"seeds":[1,3,6],"note":66,"rank":32.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6363410730956222,"sd":0.0054684058538235535,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8458290842329947,"sd":0.003848112410087937,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.8586691486947062,"sd":0.005799540928475178,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.8335476913375498,"sd":0.0005898512211733269,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.8282752941435069,"sd":0.001596617057856405,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8469177275496607,"sd":0.0012899429763727117,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7909767404476152,"sd":0.001846125581246304,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7859562068560786,"sd":0.0014228557856339281,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.8212231887113385,"sd":0.0005653760634506414,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.8412085815248863,"sd":9.838380080280182e-05,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8521622808581095,"sd":0.0015779151413086827,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8053439775556949,"sd":0.002297021976485285,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"orthrus-4track","mean":0.7650643132850632,"sd":0.009685359453708312,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"rinalmo","mean":0.8606113006467733,"sd":0.0018860069300367895,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"rnabert","mean":0.112744031485138,"sd":0.0011028987421062518,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.8278486118411359,"sd":0.004403602168358244,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"rnafm","mean":0.5575148375533874,"sd":0.008650428047503848,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"splicebert","mean":0.5799625423633262,"sd":0.004475565565491083,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false}]},{"id":"regulatory_activity_k562","benchmark":"ours_regression","name":"Regulatory Activity Prediction","modality":"DNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8655403167410527,"sd":0.005921217667211145,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6265967634653198,"sd":0.005180511084572282,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.7516556423532496,"sd":0.005588908770331965,"seeds":[1,2,3],"note":58,"rank":26.0,"top_score":false},{"model":"borzoi","mean":0.8538007791715996,"sd":0.0005908820828944844,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8108054084183843,"sd":0.003779543766162609,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8040371195733366,"sd":0.0021361695838684005,"seeds":[1,5,6],"note":67,"rank":11.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6520389973834477,"sd":0.008125492260598104,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-3","mean":0.3136429037478983,"sd":0.006357823097274724,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.31340947926136636,"sd":0.0031258551916237987,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.3103731671158827,"sd":0.004877925571183703,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-6","mean":0.3083051223971382,"sd":0.006494840407300939,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert2","mean":0.6905988708306993,"sd":0.045793735123105674,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"enformer","mean":0.8544554541234051,"sd":0.0033562873932466036,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8040480079713823,"sd":0.002152835486970442,"seeds":[1,2,3],"note":59,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7276833671542936,"sd":0.007224937761674899,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.7715949424831133,"sd":0.0025755592225339993,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7541398743745781,"sd":0.005624831286322759,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7536597761500723,"sd":0.0054252723338997985,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7825110552462848,"sd":0.0017557428832180447,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7645268827623752,"sd":0.009490483554191848,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8058760919768972,"sd":0.005559959168260064,"seeds":[1,4,5],"note":68,"rank":8.0,"top_score":false},{"model":"phylogpn","mean":0.6431830026796006,"sd":0.006540175615898517,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"grover","mean":0.7650597966918301,"sd":0.0039045981071040106,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7646492931908816,"sd":0.01037361508986496,"seeds":[1,2,3],"note":10,"rank":21.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7105791558025567,"sd":0.08093252714627774,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6339306968539304,"sd":0.004618394984963367,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8008624662028221,"sd":0.006630629296705653,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.8140537388123672,"sd":0.0010300763862092393,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.792963258314136,"sd":0.001375752337726066,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7864799227288484,"sd":0.002149919914621129,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8052465578831239,"sd":0.00418116135358558,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.7465338356023269,"sd":0.0013851476891884721,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7418185045761102,"sd":0.001406119495477877,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7825878013288001,"sd":0.00528904328205846,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7996049239554684,"sd":0.0028554155466329607,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8091386451253214,"sd":0.0040054161999435,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7642613649974103,"sd":0.0017151588521450377,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"orthrus-4track","mean":0.6834986747276672,"sd":0.037118857417123376,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.8093858177539426,"sd":0.0013846466198930913,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.11824661251917824,"sd":0.0016908847739791407,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.7907536824964175,"sd":0.004344133179246302,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"rnafm","mean":0.5374404990281739,"sd":0.0026570648509000845,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"splicebert","mean":0.5550341442178025,"sd":0.006670363376088472,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false}]},{"id":"stability","benchmark":"ours_regression","name":"Protein Stability Prediction","modality":"Protein","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5375317434179406,"sd":0.042413125677172085,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"baseline_cnn","mean":0.554858303052755,"sd":0.032150248190278646,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"baseline_lstm","mean":0.5524962057289592,"sd":0.07615272771875793,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"borzoi","mean":0.5721646528482383,"sd":0.07799002166370997,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5696257059313848,"sd":0.03205060444305742,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6632343276244115,"sd":0.04706266549008048,"seeds":[1,5,6],"note":69,"rank":2.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6091151815600617,"sd":0.05358701145963618,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"dnabert-3","mean":0.6577431710758476,"sd":0.0350462202238214,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"dnabert-4","mean":0.5782478807223467,"sd":0.01795757305681849,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-5","mean":0.6336817301613821,"sd":0.0544470998627361,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"dnabert-6","mean":0.6094946681878303,"sd":0.032008112919847044,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert2","mean":0.4133403934572082,"sd":0.028068932620059498,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.27642551507664975,"sd":0.036686526569901144,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"evo2_1b_base","mean":0.563285907778117,"sd":0.04838259492623402,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4666415224499453,"sd":0.03517780570507033,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.501660156936585,"sd":0.0804696244566526,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5518549836728202,"sd":0.01025303489812643,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5418573663904486,"sd":0.02688772294173381,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5614917565003652,"sd":0.09216884452893086,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5640734357164201,"sd":0.07594087204518456,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5768041927423831,"sd":0.07050675091099007,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"phylogpn","mean":0.5414479126218719,"sd":0.03820686543719108,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.48507412373581954,"sd":0.014553422930477695,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6015749675960557,"sd":0.039713217807404985,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6137911158126094,"sd":0.06337698954473533,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5116872153743861,"sd":0.05391684117470591,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6158357890960093,"sd":0.04099672195176926,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6216469921618377,"sd":0.041400486841869824,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6127750514923639,"sd":0.039135589203586646,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5527515764144166,"sd":0.04031880426321709,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5316031819164284,"sd":0.06827896835279686,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.4769501192600638,"sd":0.05136900701474023,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.4983115475685014,"sd":0.02993071645946554,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5621865368628675,"sd":0.04348353053335045,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5309459120820083,"sd":0.025425006466398305,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5353321091880445,"sd":0.0418415009585911,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5967371726329149,"sd":0.09312870339856866,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"orthrus-4track","mean":0.36405056641976824,"sd":0.08055347271102846,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.6155372872807159,"sd":0.02278936352165126,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.6272832451245305,"sd":0.04801137482654444,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"rnaernie","mean":0.5724111185171771,"sd":0.04528640670327802,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"rnafm","mean":0.6067146930045434,"sd":0.09121509515275654,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"splicebert","mean":0.699197532555361,"sd":0.018843137560157104,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true}]},{"id":"ta_athaliana","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5044044755029494,"sd":0.006188386460794371,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.39490571337249325,"sd":0.02532800582817894,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.4589936319775825,"sd":0.0056782510330139055,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"borzoi","mean":0.48584729283240824,"sd":0.01891092378216901,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4008372973515813,"sd":0.09562664362698342,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.45087412958286405,"sd":0.060188485395760355,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.39667290920652004,"sd":0.03159768014347248,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.42225480451344577,"sd":0.03225647421473267,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-4","mean":0.44204920582097335,"sd":0.017187058023010078,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-5","mean":0.41086259057517155,"sd":0.037973998675844084,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.4258611268668628,"sd":0.016814422355214336,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.49793883621910323,"sd":0.012732051345781065,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"enformer","mean":0.45153508639468304,"sd":0.004157790305250317,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5316576033662832,"sd":0.0053565471697662155,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.44074578030753236,"sd":0.011142229674758083,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.46586258086158955,"sd":0.005784369889424048,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.42329822291562147,"sd":0.029622584245167692,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.4689549122878453,"sd":0.01074483831805627,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.4947903214921863,"sd":0.01238885026668525,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.436550760777435,"sd":0.026172869614936988,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5912731571780995,"sd":0.004086251728808486,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.4536718057058442,"sd":0.016104946048038183,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.4714114852435149,"sd":0.01586162595430534,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.47979043591397524,"sd":0.01611900087166488,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4965591457809963,"sd":0.009056041905654282,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.46278773070533735,"sd":0.005585165946463704,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5444732928571618,"sd":0.011613797580626015,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5500915633743771,"sd":0.01819371406631384,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.531080750267276,"sd":0.013589983033117582,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5123937505070244,"sd":0.010900213246836913,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5547386132504347,"sd":0.003334185408137814,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5000633636222589,"sd":0.005271465750379125,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.49474827875092536,"sd":0.007807140520918479,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5307384178985397,"sd":0.007214774626055286,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5298457822652839,"sd":0.009478620159044195,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.529263346840023,"sd":0.02325570734640185,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5262197857490548,"sd":0.009190529970849825,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.46179674628247974,"sd":0.004293487431780255,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"rinalmo","mean":0.5606040302748108,"sd":0.005071645125529163,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"rnabert","mean":0.3025436349545241,"sd":0.008637759177679942,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.49058687428946834,"sd":0.011983186052855031,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"rnafm","mean":0.4829891513408299,"sd":0.010971726344999716,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"splicebert","mean":0.5082804829561454,"sd":0.009855194234909073,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false}]},{"id":"ta_dmelanogaster","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.4951012277210543,"sd":0.007060389592116441,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"baseline_cnn","mean":0.33786242686947976,"sd":0.00824556931498329,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.453231462855451,"sd":0.006863162020225646,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"borzoi","mean":0.4667505875775842,"sd":0.005196597256093038,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4205924454934606,"sd":0.09973002662770317,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3920683005570211,"sd":0.08590504043003902,"seeds":[1,2,5],"note":0,"rank":40.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.2302089831680335,"sd":0.14149492439941014,"seeds":[1,4,6],"note":70,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.4624439237179048,"sd":0.022927209908406316,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-4","mean":0.4419128330796083,"sd":0.013732836186341565,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"dnabert-5","mean":0.4425565309961164,"sd":0.02613623126076121,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert-6","mean":0.4073430039608239,"sd":0.010790817807548253,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.4419627036425153,"sd":0.02601287186385674,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.4443036740302975,"sd":0.012846026172444012,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"evo2_1b_base","mean":0.43039681174972794,"sd":0.09027899345268901,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4372074536873927,"sd":0.02298817452608934,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.4276279730728114,"sd":0.026760738134498918,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.4071242758771742,"sd":0.014522768354176329,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.42426245559596293,"sd":0.028117051113367195,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.46400057200491024,"sd":0.03292132656646226,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.42847407517267055,"sd":0.016775372470561725,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6199515970649552,"sd":0.0005182884000637487,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.4309715012413145,"sd":0.01635502469564138,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"grover","mean":0.4660949501777573,"sd":0.006765149982239084,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"hyena_large_1m","mean":0.41639155011049195,"sd":0.009988437127297758,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3968096313523335,"sd":0.034437961595465444,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5034584352369508,"sd":0.012440545658335782,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5609429528998985,"sd":0.01859434168689744,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.631973355362259,"sd":0.010801864751075857,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.535589906016417,"sd":0.022287501163150338,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5230035702337281,"sd":0.013390976193159177,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5713240384304911,"sd":0.011968189776969333,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.472981687436048,"sd":0.01219834176293428,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5047025603307896,"sd":0.004815738571492737,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5482262437518866,"sd":0.004309806682911156,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5628007748338276,"sd":0.010121460928161316,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5567485914331579,"sd":0.00835981946989425,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5315455125723502,"sd":0.014867824926962953,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"orthrus-4track","mean":0.43653099580853955,"sd":0.01410766699573237,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.5556307654911854,"sd":0.029389793562842322,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.31250709787707837,"sd":0.009654365856945377,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.4677689969582944,"sd":0.002863091675784487,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"rnafm","mean":0.45286550533156206,"sd":0.01676478634448876,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.5187245630745356,"sd":0.005784062180483975,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false}]},{"id":"ta_ecoli","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6085852905804593,"sd":0.0019738128650567837,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"baseline_cnn","mean":0.441166484630139,"sd":0.04565432620326295,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"baseline_lstm","mean":0.5289722429301372,"sd":0.01346282252574795,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.5952724846070983,"sd":0.003307740317712172,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4091884229337313,"sd":0.0657630163344162,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5686745702626622,"sd":0.06576110649268306,"seeds":[1,2,4],"note":0,"rank":25.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.3599075819296084,"sd":0.011074793054349384,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.5587855183413402,"sd":0.017336820991360354,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-4","mean":0.5326598673173756,"sd":0.02707229637560379,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.5263093865801357,"sd":0.03657055936242397,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.5575455828089241,"sd":0.00572457091758645,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert2","mean":0.33949840424909156,"sd":0.12118969583199189,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"enformer","mean":0.5236870991474626,"sd":0.013768269914293727,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5900535656086344,"sd":0.03280390913204299,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5235809972260691,"sd":0.002701763028267566,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6033353310498583,"sd":0.0056168392354577455,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5942713773490194,"sd":0.01586816884505494,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5647064480274955,"sd":0.028304466245508197,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5831451065955122,"sd":0.018112268609600947,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5885464945033004,"sd":0.025585063830714197,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6397159281379383,"sd":0.021848422110626844,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.5492379821551743,"sd":0.017452647675977616,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"grover","mean":0.6039548780317533,"sd":0.0017634092830919321,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5546736313421395,"sd":0.011425494880041702,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.578909055464312,"sd":0.02158523077877519,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5134978788886558,"sd":0.0787614759761071,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6203710297665207,"sd":0.028186644638834423,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7134132455364125,"sd":0.08352530625191276,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.6330576659836656,"sd":0.02644374092240623,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5931586046018488,"sd":0.004842635763776852,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6250084475651335,"sd":0.02366595443539482,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.563716443678861,"sd":0.015877613123764008,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5824265037366304,"sd":0.004586232866441649,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5902370305186939,"sd":0.0064797060924156715,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6189869293758572,"sd":0.03270729102692463,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6051151276637259,"sd":0.003380573326823288,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6123438582607518,"sd":0.012365621850285892,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"orthrus-4track","mean":0.579434987495382,"sd":0.0010853071789216648,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rinalmo","mean":0.661483521421733,"sd":0.00907149882599028,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"rnabert","mean":0.267952654688402,"sd":0.01787353405419658,"seeds":[1,2,6],"note":6,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.5742623246277819,"sd":0.021065793241954796,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.5189956855115517,"sd":0.07246144834799391,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"splicebert","mean":0.5851032884737659,"sd":0.02042010265893371,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false}]},{"id":"ta_hsapiens","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.4098133114350515,"sd":0.007634454336138755,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"baseline_cnn","mean":0.09460555148935235,"sd":0.05512614386140103,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.2631464442919287,"sd":0.011265548403198137,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"borzoi","mean":0.3285529034004189,"sd":0.00908717845680803,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.319969765236261,"sd":0.0574888007238847,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.3300276919840476,"sd":0.010042665821944992,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.053570307731639726,"sd":0.04235394673362458,"seeds":[1,4,5],"note":70,"rank":42.0,"top_score":false},{"model":"dnabert-3","mean":0.2402429904909544,"sd":0.021330651529445653,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.26034505021890636,"sd":0.03599607513818502,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-5","mean":0.24325518382178904,"sd":0.016049884700310577,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.23448908672429272,"sd":0.023133709005180452,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert2","mean":0.025559810690934825,"sd":0.06250172343972223,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"enformer","mean":0.2713244392074277,"sd":0.02120341226304916,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3699979578594973,"sd":0.041241464922689076,"seeds":[1,3,4],"note":0,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.22746688086920439,"sd":0.037175822755609655,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.3410758856027291,"sd":0.011900883345928823,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3380044521722259,"sd":0.0058248864087903415,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.3153898420915385,"sd":0.015982893446929016,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.31367696448508814,"sd":0.07570276583355977,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3148345188760461,"sd":0.019144299112720246,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.4584882051005556,"sd":0.02476168703855246,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.38398345516590277,"sd":0.0017413756912665129,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"grover","mean":0.29054775032688296,"sd":0.037447437548914914,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3009149290482323,"sd":0.012577595769683867,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.32567624363678643,"sd":0.009605505849452195,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2705558167729228,"sd":0.017740772278718408,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.36344589254696585,"sd":0.019156989854450645,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.41805659700256087,"sd":0.021643096849248197,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.41721882578218833,"sd":0.010543486849407547,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.39457504953179073,"sd":0.030651546144546564,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.4319872032740064,"sd":0.03365074035057282,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.32000426603318666,"sd":0.04995239659277498,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3476967193465117,"sd":0.075596071448706,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.4099762248287488,"sd":0.019229145822351672,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.37051741254738957,"sd":0.01953997813768977,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.40280208492298347,"sd":0.010238569730551669,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.41184359954278227,"sd":0.007869245389974777,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"orthrus-4track","mean":0.28444007731849413,"sd":0.022442299423057196,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rinalmo","mean":0.377510890653557,"sd":0.016329060267182786,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"rnabert","mean":0.17747164368897492,"sd":0.015771776907236825,"seeds":[1,7,8],"note":71,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.24161377505398254,"sd":0.09755258153398733,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"rnafm","mean":0.2233503043607161,"sd":0.014415091977087185,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"splicebert","mean":0.3922788104463261,"sd":0.01081367452258329,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false}]},{"id":"ta_hvolcanii","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.3267483806895993,"sd":0.011631472102935568,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"baseline_cnn","mean":0.2823833365117087,"sd":0.04017336850258285,"seeds":[1,2,4],"note":0,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.3039772431912541,"sd":0.013331551960787277,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.30969562552047375,"sd":0.008377727108829173,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.3026031147524366,"sd":0.05629036168256694,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.35656111796191287,"sd":0.01999624618594604,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.27681488284511785,"sd":0.021587442972316042,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-3","mean":0.36249737764001794,"sd":0.01750069235048367,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert-4","mean":0.34399143545180766,"sd":0.02240306514720717,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"dnabert-5","mean":0.35198714224057764,"sd":0.015272849045311015,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"dnabert-6","mean":0.36089083019174717,"sd":0.012231542487058787,"seeds":[1,2,5],"note":15,"rank":19.0,"top_score":false},{"model":"dnabert2","mean":0.03923590991804009,"sd":0.0314331320014735,"seeds":[1,3,5],"note":72,"rank":43.0,"top_score":false},{"model":"enformer","mean":0.3071782736079744,"sd":0.0271587004882791,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3746402120355487,"sd":0.019669996868082733,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.3616832781221322,"sd":0.03258961338784507,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.37829502508621676,"sd":0.01111361263400825,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3209305736149479,"sd":0.03488871767189603,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.3555425786645707,"sd":0.04469734431007728,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.35040894095849984,"sd":0.02955085767157053,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.31833964068173176,"sd":0.03173601706772819,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.29251077818517734,"sd":0.040243880530345603,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"phylogpn","mean":0.36204070782207487,"sd":0.028830958681928472,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"grover","mean":0.3524985390691467,"sd":0.08040746911183222,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_large_1m","mean":0.38322178904116533,"sd":0.005149227770685562,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.40346292294381536,"sd":0.021769956187896868,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.3054527376631287,"sd":0.028354812580398344,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.3939168460551243,"sd":0.018930160327108876,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.3983494327280672,"sd":0.007796838581210596,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.4138339408960945,"sd":0.015118311380852019,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.3843612415085449,"sd":0.021734598179786076,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.39924225694813026,"sd":0.022601432388172506,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.3550464357284411,"sd":0.026220258040299736,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.37648244337208253,"sd":0.041060351270074666,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3891744238591514,"sd":0.04327276872979989,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3798421727689876,"sd":0.017716436275221577,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.40528697297808375,"sd":0.013356970648181223,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.39410735151670445,"sd":0.009916337515870094,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"orthrus-4track","mean":0.31703216745295576,"sd":0.019566731363786012,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.3913448935001964,"sd":0.02652452868978106,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.21746880916543895,"sd":0.02322585385691138,"seeds":[1,2,6],"note":6,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.3353685918248246,"sd":0.0553824860804606,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"rnafm","mean":0.31015283208321903,"sd":0.011737460164849382,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.32839055663992683,"sd":0.041349096880287914,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false}]},{"id":"ta_ppastoris","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5837597246495116,"sd":0.01271258013192567,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"baseline_cnn","mean":0.4920942909684653,"sd":0.003481625589474194,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"baseline_lstm","mean":0.42762938346074875,"sd":0.009319470586002378,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.5617915518852147,"sd":0.01963584221272838,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5489713082227626,"sd":0.02092636486636307,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5657004154356008,"sd":0.0035921330805999945,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.19847349694784375,"sd":0.03019995697593239,"seeds":[2,4,5],"note":73,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.4037609565908553,"sd":0.012512016272534,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-4","mean":0.39526360108628467,"sd":0.005382232985628602,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"dnabert-5","mean":0.42728651304075216,"sd":0.014557182957461891,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-6","mean":0.4243328355092522,"sd":0.016897519197490326,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert2","mean":0.4455153481239478,"sd":0.057994685234722315,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"enformer","mean":0.4890748015780007,"sd":0.017784665348216322,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6013767461182808,"sd":0.016083236161693614,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5360213131755579,"sd":0.006759690670626895,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.56031608232608,"sd":0.015403388216588947,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.524824301102644,"sd":0.008011705047082273,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5094277301493194,"sd":0.012610742754128828,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.563949490331196,"sd":0.01376934066281449,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5601557738908753,"sd":0.006838828597242732,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6284315815122925,"sd":0.009558088590804184,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.5553322292337817,"sd":0.010985624119119495,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"grover","mean":0.5364113099139255,"sd":0.013103952373492508,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5319346552835998,"sd":0.007477666263344718,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5400523750204133,"sd":0.0024787244658160904,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5364072828475844,"sd":0.010295265113810473,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6022036307546214,"sd":0.016488362724079893,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6142028189719381,"sd":0.010089099843630246,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6256888903080338,"sd":0.02044872678413703,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.587085375144312,"sd":0.0037529141090553468,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5995928211424103,"sd":0.021180893655368595,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5729064778050305,"sd":0.01126365328478623,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5650137912337461,"sd":0.019580146494163703,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5767254740812771,"sd":0.011720239125512266,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6157996056656656,"sd":0.0014741092556082092,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6106872381387993,"sd":0.022165417137817188,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6091572581155632,"sd":0.006148934405228183,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"orthrus-4track","mean":0.48077524415750944,"sd":0.008488212195827774,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.606707013248054,"sd":0.014648819357342968,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.33336343578536287,"sd":0.009894197220913043,"seeds":[1,5,12],"note":74,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.5260513621455526,"sd":0.025516110545413258,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rnafm","mean":0.47020149982986853,"sd":0.011017421141154579,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"splicebert","mean":0.5472064278505478,"sd":0.014894132417558159,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false}]},{"id":"ta_scerevisiae","benchmark":"ours_regression","name":"Transcript and Protein Abundance Prediction","modality":"RNA","metric":"spearman","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.5609605341446994,"sd":0.0399976571327543,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"baseline_cnn","mean":0.3674888902708984,"sd":0.026791621967552155,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.5087914580967468,"sd":0.009827647130878284,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"borzoi","mean":0.5778098224422553,"sd":0.008401010315961495,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5327576500023324,"sd":0.04006588921633688,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6017054926609134,"sd":0.01998983307479817,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.32863605389623046,"sd":0.03638312843155423,"seeds":[1,5,6],"note":35,"rank":42.0,"top_score":false},{"model":"dnabert-3","mean":0.505042359343501,"sd":0.01338513920196636,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-4","mean":0.47993714980689245,"sd":0.04291428302891525,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.5040538953275527,"sd":0.015973606149530112,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"dnabert-6","mean":0.5348045433854461,"sd":0.00883532921677476,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert2","mean":0.3621865959118836,"sd":0.007534275190532511,"seeds":[1,6,8],"note":75,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.4968556205077242,"sd":0.025603428547628076,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"evo2_1b_base","mean":0.49098225214062896,"sd":0.11834013028323119,"seeds":[1,2,4],"note":0,"rank":35.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.5089736177333627,"sd":0.01163064656572471,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5965957216201111,"sd":0.017017582405362303,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5206549005793156,"sd":0.048215500926515165,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5100932104981232,"sd":0.005095511149021974,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6052539667007101,"sd":0.013660370606931734,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6067615779941639,"sd":0.009463265407062002,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6727714580129204,"sd":0.015461327366413124,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.5854615553698623,"sd":0.009395844188873199,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"grover","mean":0.5643797653180073,"sd":0.017134507750207525,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5528586788974102,"sd":0.007912070883596042,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.5294770350400777,"sd":0.04720329541781718,"seeds":[1,3,4],"note":0,"rank":27.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.4896883259158302,"sd":0.015464321055359574,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6718916669932694,"sd":0.012907560750952141,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6743861593538066,"sd":0.035842640253526854,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.6640309248085229,"sd":0.015958446882672995,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6089514484868072,"sd":0.03426307464693461,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6564240457923631,"sd":0.008107634922430365,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6233727752469904,"sd":0.012456122866466588,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.577695899010262,"sd":0.05605601208976295,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6549087577669072,"sd":0.008244496066563283,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6673787424992027,"sd":0.014914635749136975,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6670204720940535,"sd":0.020579866132354144,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6447754672800806,"sd":0.015575236377938314,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"orthrus-4track","mean":0.4909614587832332,"sd":0.013079849412332673,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"rinalmo","mean":0.6302092177453512,"sd":0.003822501726140178,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.14651604515196917,"sd":0.11985737706806036,"seeds":[1,2,4],"note":76,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.48906329796472087,"sd":0.09469313029135988,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rnafm","mean":0.5454792488304985,"sd":0.023366397563970534,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"splicebert","mean":0.5778326205703964,"sd":0.02152283759908298,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false}]},{"id":"tc_riboswitches","benchmark":"ours_regression","name":"Tc-Riboswitches Prediction","modality":"RNA","metric":"spearman","protocol_variant":"grouped_or_chronological","results":[{"model":"alphagenome","mean":-0.029124771223005445,"sd":0.055208186914347555,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"baseline_cnn","mean":-0.042731331321285065,"sd":0.022675843492852225,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"baseline_lstm","mean":0.19829165692504244,"sd":0.05105797264834637,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"borzoi","mean":-0.19883385774504236,"sd":0.00938844793800929,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":-0.01926163512115081,"sd":0.014353685077180196,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.10725506442099116,"sd":0.08813012021451937,"seeds":[1,2,4],"note":0,"rank":6.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.08382115748951136,"sd":0.04602412285063082,"seeds":[2,3,9],"note":77,"rank":9.0,"top_score":false},{"model":"dnabert-3","mean":-0.05362666513152426,"sd":0.044527430708280406,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-4","mean":-0.047228767005598894,"sd":0.0926351273068553,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-5","mean":0.14334849995697127,"sd":0.06097566289909287,"seeds":[1,3,4],"note":null,"rank":2.0,"top_score":false},{"model":"dnabert-6","mean":-0.11761416794367117,"sd":0.026763283917314674,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":-0.11789494809895257,"sd":0.04687989318898195,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"enformer","mean":0.06887290474479048,"sd":0.046089078213477117,"seeds":[3,4,5],"note":0,"rank":11.0,"top_score":false},{"model":"evo2_1b_base","mean":0.07768304629831682,"sd":0.09465398918442926,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.06053029603274708,"sd":0.05635074914835347,"seeds":[2,3,4],"note":0,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.11398054469428148,"sd":0.028268972461740507,"seeds":[1,3,4],"note":0,"rank":5.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.0057381668044232505,"sd":0.08832404845739977,"seeds":[1,2,4],"note":0,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.04328891357780648,"sd":0.07746498195013317,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.023917699713922984,"sd":0.06143882329451133,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":-0.1003649864645539,"sd":0.11999745000044448,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.05343700101050118,"sd":0.0872420840814754,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"phylogpn","mean":-0.12201636668897423,"sd":0.06460316998459452,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"grover","mean":-0.2327852435645798,"sd":0.06948885992512396,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"hyena_large_1m","mean":-0.12130487471877328,"sd":0.08366791395056035,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"hyena_medium_160k","mean":-0.04911602100025917,"sd":0.09234016017867003,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":-0.03097266884718757,"sd":0.04997818237464683,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-100m-pre","mean":-0.008303007065800701,"sd":0.07644153260962454,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"ntv3-650m-pre","mean":-0.05600864607692649,"sd":0.04016445868810323,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"ntv3-8m-pre","mean":-0.08243328891956168,"sd":0.096434223156619,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.003348023107451944,"sd":0.09608135817016263,"seeds":[2,3,4],"note":0,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.0298136767991837,"sd":0.06794336526541378,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":-0.11895005700681718,"sd":0.08469574160884512,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.04840436861753092,"sd":0.027658760714158164,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.14254935831591115,"sd":0.14574807678573806,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.0891116980052897,"sd":0.05438178818255424,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":-0.12033591032316295,"sd":0.026850502088147363,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.016817967684550142,"sd":0.08191946720981165,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":-0.07146043934110576,"sd":0.023923066261990464,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"rinalmo","mean":0.0979193785391976,"sd":0.06640496912579864,"seeds":[1,3,5],"note":0,"rank":7.0,"top_score":false},{"model":"rnabert","mean":0.04982575745441771,"sd":0.04180695663212289,"seeds":[2,3,4],"note":78,"rank":14.0,"top_score":false},{"model":"rnaernie","mean":0.11887464829739229,"sd":0.13721964371205198,"seeds":[1,4,5],"note":79,"rank":4.0,"top_score":false},{"model":"rnafm","mean":-0.08210513366996654,"sd":0.03884994791983016,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.008619462989939838,"sd":0.09091597652447599,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false}]},{"id":"tc_riboswitches_random_iid","benchmark":"ours_regression","name":"Tc-Riboswitches Prediction","modality":"RNA","metric":"spearman","protocol_variant":"random_iid","results":[{"model":"alphagenome","mean":0.6258976108239244,"sd":0.012454720290318098,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_cnn","mean":0.5977910941135103,"sd":0.04908399760406554,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"baseline_lstm","mean":0.46454113018520793,"sd":0.011800990834385907,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"borzoi","mean":0.37498249422369473,"sd":0.07769893726907086,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.42480343921915437,"sd":0.026581359904914732,"seeds":[3,4,5],"note":0,"rank":30.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.2988154595961731,"sd":0.13109654693173253,"seeds":[1,2,4],"note":0,"rank":38.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6005068216691112,"sd":0.008842521090633567,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-3","mean":0.6257349730028073,"sd":0.018170660112932977,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"dnabert-4","mean":0.6315443468197751,"sd":0.06387028235382822,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"dnabert-5","mean":0.6203516108978601,"sd":0.08328786071945853,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"dnabert-6","mean":0.6842076114451286,"sd":0.07629229207661375,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"dnabert2","mean":0.3096520765762528,"sd":0.05250078861109541,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"enformer","mean":0.05313850289720069,"sd":0.05612574038617565,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"evo2_1b_base","mean":0.30298786181139126,"sd":0.17848662419394415,"seeds":[1,10,11],"note":0,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.01923396805582849,"sd":0.1312174324521796,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.4512935639929278,"sd":0.07084726460050945,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.41714486894314395,"sd":0.10867337631886992,"seeds":[1,2,4],"note":0,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.1896358543417367,"sd":0.15700448778070375,"seeds":[1,2,4],"note":0,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3148279868256181,"sd":0.12086623807881032,"seeds":[2,3,4],"note":0,"rank":35.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.04303903655675124,"sd":0.13297024086925305,"seeds":[2,3,5],"note":8,"rank":42.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.6251319563438548,"sd":0.028928344953166037,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"phylogpn","mean":0.3779645191409898,"sd":0.06791797756572525,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"grover","mean":0.4096057970273411,"sd":0.07296358612783668,"seeds":[1,3,4],"note":0,"rank":32.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5511327332850666,"sd":0.07605006483012894,"seeds":[1,2,4],"note":0,"rank":26.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6171944533796135,"sd":0.07689284561297263,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.6093478532519183,"sd":0.029817640211322934,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6893945726372784,"sd":0.005309931022442365,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6704211928586957,"sd":0.009897233408901257,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.677750774751852,"sd":0.05654100313153586,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.6085395278218007,"sd":0.021608842111325976,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5980996367217901,"sd":0.04270812300524322,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5823492838869829,"sd":0.08077551274320184,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.6538721689203347,"sd":0.023511006039482984,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6481288766198184,"sd":0.05195575701365295,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6385206513566414,"sd":0.053678776251142674,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6007103356570866,"sd":0.024222360163597446,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6789773056284915,"sd":0.026283517520941494,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"orthrus-4track","mean":0.1523809523809524,"sd":0.029389080220961355,"seeds":[2,3,4],"note":0,"rank":40.0,"top_score":false},{"model":"rinalmo","mean":0.6478978469094769,"sd":0.09566632992004448,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"rnabert","mean":0.6439327012887035,"sd":0.028362994256430227,"seeds":[4,6,7],"note":80,"rank":11.0,"top_score":false},{"model":"rnaernie","mean":0.6638832596383684,"sd":0.050054577064742936,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"rnafm","mean":0.6974209385897381,"sd":0.019542379655349214,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"splicebert","mean":0.5438335420690283,"sd":0.05878186017883328,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false}]},{"id":"brca1_1024_coding","benchmark":"ours_var_classification","name":"BRCA1/2 SNV Functional Impact Classification","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.8538232302833875,"sd":0.026050195685099278,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"baseline_cnn","mean":0.8319511063209648,"sd":0.013874949842465064,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"baseline_lstm","mean":0.83183165278686,"sd":0.019780032254884176,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"borzoi","mean":0.7602212238755086,"sd":0.06562248234385314,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8090823342101156,"sd":0.016097895668184377,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8353869180319623,"sd":0.029092850445161588,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.782258064516129,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"dnabert-3","mean":0.8469158688034415,"sd":0.013434335189314492,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"dnabert-4","mean":0.8481900307704388,"sd":0.0068631454489611055,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"dnabert-5","mean":0.8386998547988136,"sd":0.012837336792747974,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"dnabert-6","mean":0.8544702079232631,"sd":0.010241519069291253,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"dnabert2","mean":0.7855079406407076,"sd":0.0044009065524354095,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.8183667338200922,"sd":0.016618561677598367,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8052674831334199,"sd":0.029104666076265162,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.7930617474983273,"sd":0.024832530318090972,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8308707619867317,"sd":0.012459791350087842,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.859125340455008,"sd":0.029371433381078126,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.8683145354566847,"sd":0.03314559347658052,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"gena-lm-bert-large-t2t","mean":0.8398880970403738,"sd":0.009195249999956433,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7744389605021895,"sd":0.041802656556195555,"seeds":[1,3,4],"note":null,"rank":41.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8328109357675786,"sd":0.04634523891101443,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"phylogpn","mean":0.6832358374413631,"sd":0.02208553653787432,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"grover","mean":0.818001765277972,"sd":0.04287976080194788,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"hyena_large_1m","mean":0.7956107757816029,"sd":0.012197463640989639,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.8282415648343389,"sd":0.024275269946260877,"seeds":[1,2,4],"note":81,"rank":19.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.7893364835619399,"sd":0.013610776252631861,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.8027899143264485,"sd":0.06460676773391079,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.810374885264936,"sd":0.036205708815225134,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7807121314038672,"sd":0.057326471874826763,"seeds":[2,4,5],"note":null,"rank":40.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.8525105653032611,"sd":0.006212031493417051,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8300049812350189,"sd":0.01912305107878174,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8131055701452007,"sd":0.013139633188166092,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.8056880405952361,"sd":0.012495699718965641,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7949642002849515,"sd":0.035642580634485006,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7913400966928539,"sd":0.03584628247446098,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7930540107005856,"sd":0.04710765940126809,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8131877498896849,"sd":0.023510530108251642,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.8385417719526701,"sd":0.02789736429701255,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"rinalmo","mean":0.782258064516129,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rnabert","mean":0.782258064516129,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rnaernie","mean":0.781168154917199,"sd":0.0006266655049824199,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"rnafm","mean":0.82961917166165,"sd":0.0274670905859468,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"splicebert","mean":0.8463291087002341,"sd":0.016990651296243615,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false}]},{"id":"brca1_1024_noncoding","benchmark":"ours_var_classification","name":"BRCA1/2 SNV Functional Impact Classification","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9517075848776518,"sd":0.004172350052959073,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.6969462714138782,"sd":0.09488784595386296,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"baseline_lstm","mean":0.5983857851443839,"sd":0.01947813353979091,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"borzoi","mean":0.806698568324853,"sd":0.08081866690654319,"seeds":[2,3,4],"note":0,"rank":4.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6189393784886106,"sd":0.03215853139353791,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.6339371455018374,"sd":0.028357027256992323,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6031746031746031,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":38.5,"top_score":false},{"model":"dnabert-3","mean":0.6433242380298551,"sd":0.06665381488664011,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert-4","mean":0.6724152128838694,"sd":0.018057131253666647,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"dnabert-5","mean":0.691047259926319,"sd":0.005619256211354913,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"dnabert-6","mean":0.7353249939785499,"sd":0.06475177314650475,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"dnabert2","mean":0.6120657609153959,"sd":0.04765941291891756,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.7545223429617807,"sd":0.038285701567370586,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7335522901800289,"sd":0.12763363077445436,"seeds":[1,3,4],"note":0,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6013066428591944,"sd":0.07607173037644237,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6359572643734525,"sd":0.020699091835968915,"seeds":[1,4,5],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6491252945164758,"sd":0.050616669734036376,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.656219641451162,"sd":0.053589203004561256,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6714950346495996,"sd":0.08988512990376414,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.7080554373246489,"sd":0.0805756848136699,"seeds":[1,4,5],"note":null,"rank":17.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8334191105645583,"sd":0.1354169338846089,"seeds":[1,3,5],"note":0,"rank":3.0,"top_score":false},{"model":"phylogpn","mean":0.7457195281174362,"sd":0.035615298054791474,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"grover","mean":0.7957585233422724,"sd":0.07421867554255328,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"hyena_large_1m","mean":0.5959087414343698,"sd":0.02224987659427395,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6096331727980084,"sd":0.07251827088598439,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.7140037359121081,"sd":0.0360566214363529,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7931716023479051,"sd":0.07093995157473512,"seeds":[1,4,5],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.925609718367781,"sd":0.03275396358342589,"seeds":[1,3,4],"note":0,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7077349790647799,"sd":0.08907458016169074,"seeds":[1,3,4],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.773916396116093,"sd":0.017974371385038844,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7646010381242737,"sd":0.115904532970282,"seeds":[1,4,5],"note":0,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6127118976899039,"sd":0.05591705213479404,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7418534728053713,"sd":0.08165803821124211,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6610133582189853,"sd":0.10265639982633532,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7028770138043254,"sd":0.1285243931162094,"seeds":[1,3,4],"note":null,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7747061362907939,"sd":0.033012630611094565,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6037670344161735,"sd":0.09737179713573514,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"orthrus-4track","mean":0.7462556416357962,"sd":0.10169012976057983,"seeds":[2,3,4],"note":0,"rank":11.0,"top_score":false},{"model":"rinalmo","mean":0.5984830248880204,"sd":0.00812605196004823,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"rnabert","mean":0.6031746031746031,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":38.5,"top_score":false},{"model":"rnaernie","mean":0.6921437480285363,"sd":0.046626724544281364,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"rnafm","mean":0.6148657100629062,"sd":0.040848325282506634,"seeds":[2,3,4],"note":null,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.6551747519616311,"sd":0.029423363217489274,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false}]},{"id":"brca2_1024_coding","benchmark":"ours_var_classification","name":"BRCA1/2 SNV Functional Impact Classification","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9205622594330327,"sd":0.0009877988959221455,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"baseline_cnn","mean":0.8984649474173539,"sd":0.007303500804301479,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"baseline_lstm","mean":0.9152431671097846,"sd":0.0014562872868789717,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"borzoi","mean":0.9020987721433134,"sd":0.0081584816379638,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8735461403719188,"sd":0.011386019219356641,"seeds":[3,4,5],"note":81,"rank":33.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8855035843716526,"sd":0.01076962167323572,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.865979381443299,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.8686097229072439,"sd":0.007488925753841407,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"dnabert-4","mean":0.874207103940147,"sd":0.014250833395118262,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"dnabert-5","mean":0.892326568853863,"sd":0.02648418573851502,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"dnabert-6","mean":0.8792244448044255,"sd":0.013059945457477436,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert2","mean":0.8703345161285837,"sd":0.004069058714167604,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"enformer","mean":0.886900879371959,"sd":0.003722310174736389,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"evo2_1b_base","mean":0.8789177996823705,"sd":0.008741670171641541,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.9024106044202934,"sd":0.002605528115577351,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.8993106380548161,"sd":0.012001529398515638,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.8962535284128336,"sd":0.0021138109028443833,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.8928809169413114,"sd":0.009082497002419802,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.899213726260953,"sd":0.011965969484807175,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.8669225048729562,"sd":0.010089614676729808,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9174640872425648,"sd":0.015507591224803824,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"phylogpn","mean":0.9170046786345264,"sd":0.0012514733887442587,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"grover","mean":0.8865832540811609,"sd":0.0054130972024754925,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"hyena_large_1m","mean":0.8707146223630441,"sd":0.01011997241518411,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.8765432367080117,"sd":0.014059770669288315,"seeds":[1,5,6],"note":82,"rank":30.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.880248272161868,"sd":0.018802369029611184,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9121306191834796,"sd":0.017999963727079165,"seeds":[1,3,4],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9146102047496103,"sd":0.008416522135225896,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9263260658531102,"sd":0.004949048310267501,"seeds":[1,3,4],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.9089679577342256,"sd":0.005313881331072878,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.8983246962846204,"sd":0.0070146570563395716,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.8912605542287925,"sd":0.015844515419405233,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.8917220650685356,"sd":0.012063256206750209,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.9039399038070212,"sd":0.0404544884952057,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.8357667831610994,"sd":0.07461030714071777,"seeds":[1,4,5],"note":null,"rank":43.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.8763723896773116,"sd":0.06513996731208396,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8663097759430689,"sd":0.04753412728718881,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"orthrus-4track","mean":0.8934930392620858,"sd":0.03962806573510847,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"rinalmo","mean":0.865979381443299,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnabert","mean":0.865979381443299,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.8658259460516837,"sd":0.00026575789395694865,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"rnafm","mean":0.898769923782555,"sd":0.010996608842071317,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"splicebert","mean":0.9227284879572646,"sd":0.009473285790511548,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false}]},{"id":"brca2_1024_noncoding","benchmark":"ours_var_classification","name":"BRCA1/2 SNV Functional Impact Classification","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.9373926685627322,"sd":0.002864586982938877,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"baseline_cnn","mean":0.9070293222167376,"sd":0.021692480568213178,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"baseline_lstm","mean":0.8839130604448507,"sd":0.01349589211102428,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"borzoi","mean":0.9125388136664139,"sd":0.05713578342526687,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.8935323901830079,"sd":0.03881209598744029,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.8319823447253908,"sd":0.041513918025785496,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.8202247191011236,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.5,"top_score":false},{"model":"dnabert-3","mean":0.9435628389219625,"sd":0.007145635660071333,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"dnabert-4","mean":0.8690606518655294,"sd":0.04782150279358679,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"dnabert-5","mean":0.9228458505372873,"sd":0.011711647752255241,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"dnabert-6","mean":0.8679934292170601,"sd":0.008636846164093884,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert2","mean":0.8544703994280175,"sd":0.04813539206895649,"seeds":[1,4,6],"note":null,"rank":33.0,"top_score":false},{"model":"enformer","mean":0.9273352266772129,"sd":0.014405522972362634,"seeds":[1,3,4],"note":0,"rank":10.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7925045943236732,"sd":0.0282797003531179,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.8382017126672857,"sd":0.02209839737329238,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.894784987958188,"sd":0.03885235300787333,"seeds":[1,3,4],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.8713732786552836,"sd":0.03960815188874574,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.8774460775676108,"sd":0.023409133366095954,"seeds":[1,4,5],"note":null,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.9032075285436242,"sd":0.06778130711356876,"seeds":[1,3,4],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.8419192228812854,"sd":0.061989397822350384,"seeds":[1,4,5],"note":null,"rank":34.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.9675657769001372,"sd":0.017207272172504253,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.9407376175054541,"sd":0.04753361299886308,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"grover","mean":0.9350386348188439,"sd":0.0212634284328377,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"hyena_large_1m","mean":0.8854365895126596,"sd":0.0384630522862907,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.8706299833858364,"sd":0.05557576827226683,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.8323646523082189,"sd":0.08023693048158934,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.9085290622665281,"sd":0.016229635070161584,"seeds":[1,3,4],"note":null,"rank":15.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.9365628200311781,"sd":0.006227869358286622,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.9115791418876784,"sd":0.006682837557333674,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.8899964699989648,"sd":0.011693288072188958,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.9686153162850684,"sd":0.01203270640189984,"seeds":[1,4,5],"note":null,"rank":1.0,"top_score":true},{"model":"nucleotide-transformer-500m-1000g","mean":0.8400299752617512,"sd":0.007943795224605673,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.9209861927099668,"sd":0.023587066096439954,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.8919726854974522,"sd":0.04117175930233888,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.9005566642542798,"sd":0.028797830894834963,"seeds":[1,2,4],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.9408772999185715,"sd":0.050349652262943585,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.8559859796602316,"sd":0.03678117531593153,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"orthrus-4track","mean":0.7731098728044952,"sd":0.0679966193020127,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rinalmo","mean":0.8347170621618261,"sd":0.025403549247867512,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"rnabert","mean":0.8202247191011236,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.5,"top_score":false},{"model":"rnaernie","mean":0.869074067372948,"sd":0.03406184084953888,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"rnafm","mean":0.8868360322013138,"sd":0.021377166313961343,"seeds":[2,3,4],"note":null,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.9401966641966961,"sd":0.02161764148394168,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false}]},{"id":"breast_disease_type_classification_512","benchmark":"ours_var_classification","name":"Genetic Disease Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"gene_disjoint_v1","results":[{"model":"alphagenome","mean":0.25412153678262933,"sd":0.007306055910670435,"seeds":[1,2,3],"note":83,"rank":9.0,"top_score":false},{"model":"baseline_cnn","mean":0.24880338895473492,"sd":0.005964649124527702,"seeds":[1,2,3],"note":83,"rank":21.0,"top_score":false},{"model":"baseline_lstm","mean":0.2500146845713467,"sd":0.005076469092576486,"seeds":[1,2,3],"note":83,"rank":14.0,"top_score":false},{"model":"borzoi","mean":0.2554893061702447,"sd":0.006990029819174851,"seeds":[1,2,3],"note":83,"rank":7.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.23975953120437168,"sd":0.01263596819596608,"seeds":[1,2,3],"note":83,"rank":37.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.2600904663327191,"sd":0.016211919511257472,"seeds":[1,2,3],"note":83,"rank":2.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":83,"rank":15.5,"top_score":false},{"model":"dnabert-3","mean":0.2469157263652836,"sd":0.008172410828797608,"seeds":[1,2,3],"note":83,"rank":23.0,"top_score":false},{"model":"dnabert-4","mean":0.2418623062744605,"sd":0.01575288722508062,"seeds":[1,2,3],"note":83,"rank":34.0,"top_score":false},{"model":"dnabert-5","mean":0.2533536829557984,"sd":0.0023845438149792675,"seeds":[1,2,3],"note":83,"rank":11.0,"top_score":false},{"model":"dnabert-6","mean":0.2401595575941517,"sd":0.008827116874757675,"seeds":[1,2,3],"note":83,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.24928919453573348,"sd":0.002162750425692552,"seeds":[1,2,3],"note":83,"rank":20.0,"top_score":false},{"model":"enformer","mean":0.2565970681459639,"sd":0.019156307514162692,"seeds":[1,2,3],"note":83,"rank":6.0,"top_score":false},{"model":"evo2_1b_base","mean":0.25123409091723164,"sd":0.005376235737999973,"seeds":[1,2,3],"note":84,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.24478630724886782,"sd":0.0048412320454927875,"seeds":[1,2,3],"note":83,"rank":26.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.25366957448235256,"sd":0.011906331012926949,"seeds":[1,2,3],"note":83,"rank":10.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.23899526862552115,"sd":0.003087173814762257,"seeds":[1,2,3],"note":83,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.23613480442604623,"sd":0.0037981763405045333,"seeds":[1,2,3],"note":83,"rank":40.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.24100031433177938,"sd":0.001129243047311494,"seeds":[1,2,3],"note":83,"rank":35.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2485764992172304,"sd":0.004079612291136918,"seeds":[1,2,3],"note":83,"rank":22.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.23444994635120023,"sd":0.017655271848690184,"seeds":[1,2,3],"note":83,"rank":42.0,"top_score":false},{"model":"phylogpn","mean":0.2596007912160215,"sd":0.03927512147091061,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"grover","mean":0.24303778861529898,"sd":0.006296429957997305,"seeds":[1,2,3],"note":83,"rank":31.0,"top_score":false},{"model":"hyena_large_1m","mean":0.2622222982130629,"sd":0.004776865081127712,"seeds":[1,2,3],"note":83,"rank":1.0,"top_score":true},{"model":"hyena_medium_160k","mean":0.2573711317468378,"sd":0.005292375494633197,"seeds":[1,2,3],"note":83,"rank":4.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2496232484999998,"sd":0.0019863045807403876,"seeds":[1,2,3],"note":83,"rank":17.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.2457739559342275,"sd":0.003770777315432367,"seeds":[1,2,3],"note":83,"rank":25.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.2542754592492315,"sd":0.009368242360917375,"seeds":[1,2,3],"note":83,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.24386285683768463,"sd":0.01415056075682911,"seeds":[1,2,3],"note":83,"rank":30.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2425360084972915,"sd":0.003542856613938793,"seeds":[1,2,3],"note":83,"rank":33.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.25702942334472706,"sd":0.010301341308554693,"seeds":[1,2,3],"note":83,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.24424363986362885,"sd":0.0062589734811076565,"seeds":[1,2,3],"note":83,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.24276852190394624,"sd":0.0017171784702308933,"seeds":[1,2,3],"note":83,"rank":32.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.24404684046443215,"sd":0.006504889979242014,"seeds":[1,2,3],"note":83,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.23890736061241144,"sd":0.008968614115104393,"seeds":[1,2,3],"note":83,"rank":39.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.23520025901544994,"sd":0.013789101243379613,"seeds":[1,2,3],"note":83,"rank":41.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.24934987979097048,"sd":0.004244134316106035,"seeds":[1,2,3],"note":83,"rank":19.0,"top_score":false},{"model":"orthrus-4track","mean":0.2445135509645607,"sd":0.0030676512151744423,"seeds":[1,2,3],"note":83,"rank":27.0,"top_score":false},{"model":"rinalmo","mean":0.2465325164747623,"sd":0.009866702728266901,"seeds":[1,2,3],"note":83,"rank":24.0,"top_score":false},{"model":"rnabert","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":83,"rank":15.5,"top_score":false},{"model":"rnaernie","mean":0.2514675745086856,"sd":0.0025353230968434403,"seeds":[1,2,3],"note":83,"rank":12.0,"top_score":false},{"model":"rnafm","mean":0.24959905507884253,"sd":0.00917870920067838,"seeds":[1,2,3],"note":83,"rank":18.0,"top_score":false},{"model":"splicebert","mean":0.23121002834531065,"sd":0.006757266591879612,"seeds":[1,2,3],"note":83,"rank":43.0,"top_score":false}]},{"id":"cardiovascular_disorders_classification_512","benchmark":"ours_var_classification","name":"Genetic Disease Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"gene_disjoint_v1","results":[{"model":"alphagenome","mean":0.21837562599253033,"sd":0.044473664368913896,"seeds":[1,2,3],"note":83,"rank":12.0,"top_score":false},{"model":"baseline_cnn","mean":0.16591754193309954,"sd":0.003281034998180143,"seeds":[1,2,3],"note":83,"rank":43.0,"top_score":false},{"model":"baseline_lstm","mean":0.19555467529733847,"sd":0.0005450410335416016,"seeds":[1,2,3],"note":83,"rank":25.0,"top_score":false},{"model":"borzoi","mean":0.17656242356705798,"sd":0.007297018810441,"seeds":[1,2,3],"note":83,"rank":34.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.18532971187420452,"sd":0.014085269287057719,"seeds":[1,2,3],"note":83,"rank":29.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.17615609518641012,"sd":0.012278666977892755,"seeds":[1,2,3],"note":85,"rank":36.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.16680992751840584,"sd":0.00024813507394784516,"seeds":[1,2,3],"note":83,"rank":42.0,"top_score":false},{"model":"dnabert-3","mean":0.210038118644816,"sd":0.019973885407188564,"seeds":[1,2,3],"note":83,"rank":15.0,"top_score":false},{"model":"dnabert-4","mean":0.20952573954419096,"sd":0.010622657236228275,"seeds":[1,2,3],"note":83,"rank":16.0,"top_score":false},{"model":"dnabert-5","mean":0.20039431424405824,"sd":0.005428234934341893,"seeds":[1,2,3],"note":83,"rank":23.0,"top_score":false},{"model":"dnabert-6","mean":0.20552761767760103,"sd":0.006370398553841508,"seeds":[1,2,3],"note":83,"rank":18.0,"top_score":false},{"model":"dnabert2","mean":0.16719269973393547,"sd":0.00015876917581461315,"seeds":[1,2,3],"note":83,"rank":40.0,"top_score":false},{"model":"enformer","mean":0.17768396622521096,"sd":0.016450277422962505,"seeds":[1,2,3],"note":83,"rank":33.0,"top_score":false},{"model":"evo2_1b_base","mean":0.18198876529646493,"sd":0.008187659849883934,"seeds":[1,2,3],"note":84,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.1869982919177787,"sd":0.007922721699307017,"seeds":[1,2,3],"note":83,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.21217148943912856,"sd":0.02495294118269832,"seeds":[1,2,3],"note":83,"rank":14.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.20580397615597001,"sd":0.006372639037130695,"seeds":[1,2,3],"note":83,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.22203363748258156,"sd":0.004947881352945374,"seeds":[1,2,3],"note":83,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.20445033368718313,"sd":0.026116656995037502,"seeds":[1,2,3],"note":83,"rank":20.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.1674490158348745,"sd":0.0011490842775927924,"seeds":[1,2,3],"note":83,"rank":39.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.28332330564186575,"sd":0.003899240226560765,"seeds":[1,2,3],"note":83,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.1711692491314851,"sd":0.00623681432197767,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"grover","mean":0.18387101829854097,"sd":0.008133847147986971,"seeds":[1,2,3],"note":83,"rank":30.0,"top_score":false},{"model":"hyena_large_1m","mean":0.1777439726152772,"sd":0.001842989800048689,"seeds":[1,2,3],"note":83,"rank":32.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.18953255742683617,"sd":0.01672951842739233,"seeds":[1,2,3],"note":83,"rank":26.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.1699617672714783,"sd":0.0012328025320572837,"seeds":[1,2,3],"note":83,"rank":38.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.23145678948455625,"sd":0.024077594403731604,"seeds":[1,2,3],"note":83,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.26076380269095634,"sd":0.005971946662444907,"seeds":[1,2,3],"note":83,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.23681543892613605,"sd":0.004613794619187419,"seeds":[1,2,3],"note":83,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.23498038719099662,"sd":0.01235031612611457,"seeds":[1,2,3],"note":83,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.2505982480262419,"sd":0.0242879755007838,"seeds":[1,2,3],"note":83,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.21675227864321833,"sd":0.005101666496135078,"seeds":[1,2,3],"note":83,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.20173364793232917,"sd":0.0061943378899913274,"seeds":[1,2,3],"note":83,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.21877559353678408,"sd":0.008521683728494822,"seeds":[1,2,3],"note":83,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.20459042476791578,"sd":0.026917165279573448,"seeds":[1,2,3],"note":83,"rank":19.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.26103891890455255,"sd":0.008957506669657634,"seeds":[1,2,3],"note":83,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.20066129214905656,"sd":0.02400414391156402,"seeds":[1,2,3],"note":83,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.17640943117403582,"sd":0.0015108439916486432,"seeds":[1,2,3],"note":83,"rank":35.0,"top_score":false},{"model":"rinalmo","mean":0.19995397149243138,"sd":0.025632688589369675,"seeds":[1,2,3],"note":83,"rank":24.0,"top_score":false},{"model":"rnabert","mean":0.18937218235308587,"sd":0.024718241074165825,"seeds":[1,2,3],"note":83,"rank":27.0,"top_score":false},{"model":"rnaernie","mean":0.16711740291244334,"sd":0.0020399644941552692,"seeds":[1,2,3],"note":83,"rank":41.0,"top_score":false},{"model":"rnafm","mean":0.2214138168723193,"sd":0.003603595648626385,"seeds":[1,2,3],"note":83,"rank":9.0,"top_score":false},{"model":"splicebert","mean":0.21859354877739998,"sd":0.015920274843303814,"seeds":[1,2,3],"note":83,"rank":11.0,"top_score":false}]},{"id":"kidney_disease_type_classification_512","benchmark":"ours_var_classification","name":"Genetic Disease Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"gene_disjoint_v1","results":[{"model":"alphagenome","mean":0.309489429905229,"sd":0.014500976488937748,"seeds":[1,2,3],"note":83,"rank":30.0,"top_score":false},{"model":"baseline_cnn","mean":0.308398226740642,"sd":0.004308772425363058,"seeds":[1,2,3],"note":83,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.3000416811600864,"sd":0.0033325785522325185,"seeds":[1,2,3],"note":83,"rank":39.0,"top_score":false},{"model":"borzoi","mean":0.31014908758948156,"sd":0.011990791106737161,"seeds":[1,2,3],"note":83,"rank":28.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.33359226560154703,"sd":0.007203311908667152,"seeds":[1,2,3],"note":83,"rank":12.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.30947604527581885,"sd":0.004540813912666413,"seeds":[1,2,3],"note":83,"rank":31.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.3333333333333333,"sd":0.0,"seeds":[1,2,3],"note":83,"rank":13.5,"top_score":false},{"model":"dnabert-3","mean":0.33968393690515436,"sd":0.009416322981198166,"seeds":[1,2,3],"note":83,"rank":5.0,"top_score":false},{"model":"dnabert-4","mean":0.33207051197860354,"sd":0.01341745732983882,"seeds":[1,2,3],"note":83,"rank":15.0,"top_score":false},{"model":"dnabert-5","mean":0.34349460093026957,"sd":0.016011614624225567,"seeds":[1,2,3],"note":83,"rank":3.0,"top_score":false},{"model":"dnabert-6","mean":0.33874788057692073,"sd":0.008628286355919595,"seeds":[1,2,3],"note":83,"rank":6.0,"top_score":false},{"model":"dnabert2","mean":0.33438098784270054,"sd":0.0039346642755702,"seeds":[1,2,3],"note":83,"rank":9.0,"top_score":false},{"model":"enformer","mean":0.30880250532000664,"sd":0.006904995246275204,"seeds":[1,2,3],"note":83,"rank":32.0,"top_score":false},{"model":"evo2_1b_base","mean":0.337166452047328,"sd":0.00026392690091023736,"seeds":[1,2,3],"note":84,"rank":8.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.30119230342792863,"sd":0.009099293399366394,"seeds":[1,2,3],"note":83,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.31570381353260185,"sd":0.015641866927137777,"seeds":[1,2,3],"note":83,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.31231279713840704,"sd":0.01152009261489386,"seeds":[1,2,3],"note":83,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.30311637702007854,"sd":0.003564353087185463,"seeds":[1,2,3],"note":83,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3289676531530454,"sd":0.013145112151976761,"seeds":[1,2,3],"note":83,"rank":16.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3401214658627561,"sd":0.011813600437770726,"seeds":[1,2,3],"note":83,"rank":4.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.3257573217009693,"sd":0.026411853247513165,"seeds":[1,2,3],"note":83,"rank":19.0,"top_score":false},{"model":"phylogpn","mean":0.3220305442322134,"sd":0.014191943535701391,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"grover","mean":0.3101426295650642,"sd":0.004834975657705837,"seeds":[1,2,3],"note":83,"rank":29.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3549692383655925,"sd":0.047339972238917886,"seeds":[1,2,3],"note":83,"rank":1.0,"top_score":true},{"model":"hyena_medium_160k","mean":0.34471587594779074,"sd":0.030408476320087748,"seeds":[1,2,3],"note":83,"rank":2.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.32767939532412393,"sd":0.011397610261568908,"seeds":[1,2,3],"note":83,"rank":18.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.333656410042568,"sd":0.013445371862364745,"seeds":[1,2,3],"note":83,"rank":11.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.31537595580939104,"sd":0.01251460484195614,"seeds":[1,2,3],"note":83,"rank":25.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3185683019603299,"sd":0.02171147931724661,"seeds":[1,2,3],"note":83,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.30555890182031475,"sd":0.003151559257674913,"seeds":[1,2,3],"note":83,"rank":36.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.29652837269015564,"sd":0.012754183162629499,"seeds":[1,2,3],"note":83,"rank":42.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.30673959720308575,"sd":0.014877617857493766,"seeds":[1,2,3],"note":83,"rank":35.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.2936619552319267,"sd":0.010027752506099332,"seeds":[1,2,3],"note":83,"rank":43.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3074691794078925,"sd":0.009486904393421272,"seeds":[1,2,3],"note":83,"rank":34.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.3162970338433808,"sd":0.009991954411087118,"seeds":[1,2,3],"note":83,"rank":23.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.3276817452893059,"sd":0.024321235474660394,"seeds":[1,2,3],"note":83,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.33717710501028164,"sd":0.02451584265745237,"seeds":[1,2,3],"note":83,"rank":7.0,"top_score":false},{"model":"orthrus-4track","mean":0.31317601042252957,"sd":0.018257741362953554,"seeds":[1,2,3],"note":83,"rank":26.0,"top_score":false},{"model":"rinalmo","mean":0.2996703064569055,"sd":0.010578640308932755,"seeds":[1,2,3],"note":83,"rank":40.0,"top_score":false},{"model":"rnabert","mean":0.3333333333333333,"sd":0.0,"seeds":[1,2,3],"note":83,"rank":13.5,"top_score":false},{"model":"rnaernie","mean":0.3343431809402275,"sd":0.001543232036744453,"seeds":[1,2,3],"note":83,"rank":10.0,"top_score":false},{"model":"rnafm","mean":0.3220097858483163,"sd":0.017234286772991016,"seeds":[1,2,3],"note":83,"rank":21.0,"top_score":false},{"model":"splicebert","mean":0.2991339444882327,"sd":0.009118264583221252,"seeds":[1,2,3],"note":83,"rank":41.0,"top_score":false}]},{"id":"pathogenicity_classification_512","benchmark":"ours_var_classification","name":"Pathogenicity Classification","modality":"DNA","metric":"ap_macro","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.3166344919986781,"sd":0.03729867556277132,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"baseline_cnn","mean":0.29615383237097764,"sd":0.0037340644778867415,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"baseline_lstm","mean":0.3122340017822722,"sd":0.004845866630117608,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"borzoi","mean":0.3559107960163627,"sd":0.0022833330401007277,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.2829325752201546,"sd":0.0019517449651212018,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.28434058632125114,"sd":0.0036334775531109693,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-3","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-5","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"dnabert2","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"enformer","mean":0.3507989612445569,"sd":0.0027867416850991204,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"evo2_1b_base","mean":0.3117476045678849,"sd":0.015683079528040936,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.2702528691417234,"sd":0.0032235676416145463,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.2703000594376385,"sd":0.004837122444159489,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.2870567949766595,"sd":0.0016624718542426122,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.283463609262821,"sd":0.0021913214837946363,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.2733963543622614,"sd":0.014837601093672801,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.45921232279830315,"sd":0.005397690933108076,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.3907529861346851,"sd":0.043268335986207766,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"hyena_large_1m","mean":0.2739704894017916,"sd":0.0013280391486237394,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.267835924542407,"sd":0.0016427003007716831,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2748922484130559,"sd":0.0027118538365287124,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.4061585211344024,"sd":0.017912922638610215,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.4419252119222841,"sd":0.001821157723881943,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.397651059482505,"sd":0.017167253282876018,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.2817626662523158,"sd":0.0036876131500800484,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.43159200493612965,"sd":0.018471183859025596,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.27392022119197645,"sd":0.006065642635517994,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.27875381491947965,"sd":0.006519693749719226,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.35711627096530124,"sd":0.03838527817414233,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.41988742891272285,"sd":0.003401237349282125,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.4459252147281401,"sd":0.017357547409971023,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3094172727603948,"sd":0.006666825385553791,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"orthrus-4track","mean":0.3050891987704046,"sd":0.021776187389435136,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"rinalmo","mean":0.25,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":39.0,"top_score":false},{"model":"rnabert","mean":0.25863013689442044,"sd":0.014947835577410848,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"rnaernie","mean":0.25008004801504835,"sd":0.00014413952355080522,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"rnafm","mean":0.29338263654061963,"sd":0.019176198788463123,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"splicebert","mean":0.4359943409849028,"sd":0.0006188219267059548,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false}]},{"id":"splicevardb_1024_exonic","benchmark":"ours_var_classification","name":"Splice Variant Effect Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.833728179377312,"sd":0.010232627529340487,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"baseline_cnn","mean":0.7455019942889433,"sd":0.012155470054786335,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"baseline_lstm","mean":0.7611642221333305,"sd":0.007371617386587467,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"borzoi","mean":0.7666627998673747,"sd":0.0047210013247546215,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.6591681278910521,"sd":0.01045352334141865,"seeds":[1,4,5],"note":81,"rank":35.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7250785949294034,"sd":0.02083608691321095,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.6526082130965594,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-3","mean":0.7136358727244885,"sd":0.045219052844803244,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"dnabert-4","mean":0.652262360130102,"sd":0.0005990349098525569,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-5","mean":0.6845563287884489,"sd":0.05634505786171469,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert-6","mean":0.7568714546538183,"sd":0.02429859212786442,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"dnabert2","mean":0.6550123302855215,"sd":0.0024090970093234637,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"enformer","mean":0.7440305297531246,"sd":0.004783906632446835,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"evo2_1b_base","mean":0.6572378084465833,"sd":0.015972283817287054,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6998494171863866,"sd":0.030133022887237817,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.704509108318874,"sd":0.043181550008854964,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.7218732737114625,"sd":0.009336401697770686,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.7353822117163533,"sd":0.004383159276986071,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7496771377853865,"sd":0.01995104954232519,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6520095250381872,"sd":0.005607466805179367,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8384426568101468,"sd":0.0055157095890044135,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.775256089330437,"sd":0.005700974838581228,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"grover","mean":0.7106584945083787,"sd":0.01958523688492921,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false},{"model":"hyena_large_1m","mean":0.6895537467426124,"sd":0.01016334286736978,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.7077848768242215,"sd":0.013154384777190182,"seeds":[1,2,6],"note":82,"rank":30.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.7225418948572977,"sd":0.009629660507056012,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7561573873286088,"sd":0.017035809155511856,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7621319947069493,"sd":0.024646874727423263,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7765273765036614,"sd":0.003337284506474655,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.7691740237720036,"sd":0.0159014885639887,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7910064348850301,"sd":0.009172602029894325,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.755766243529564,"sd":0.008546585060298987,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.7637094894320862,"sd":0.013215389105583243,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7584735121797149,"sd":0.010144313747419807,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.777106115716928,"sd":0.01694778719438854,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7995629898119103,"sd":0.023143868229869385,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.7357042172153074,"sd":0.009201770849608725,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"orthrus-4track","mean":0.7776875071856245,"sd":0.010972844805843712,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"rinalmo","mean":0.6526082130965594,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnabert","mean":0.6526082130965594,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"rnaernie","mean":0.6546708996102819,"sd":0.001815973290946765,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"rnafm","mean":0.7229167393089727,"sd":0.016661680001103622,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"splicebert","mean":0.7800683260603213,"sd":0.008866035566805217,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false}]},{"id":"splicevardb_1024_intronic","benchmark":"ours_var_classification","name":"Splice Variant Effect Prediction","modality":"DNA","metric":"ap_binary","protocol_variant":"existing","results":[{"model":"alphagenome","mean":0.6760257373140784,"sd":0.008979480687176855,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"baseline_cnn","mean":0.5064097194856211,"sd":0.010332286530434043,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"baseline_lstm","mean":0.5431264369586433,"sd":0.0013219336391750898,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"borzoi","mean":0.6252881645236461,"sd":0.003552321702416817,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5700413788449555,"sd":0.012602489380053181,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5914516920727666,"sd":0.006329817934862642,"seeds":[1,2,3],"note":null,"rank":15.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.36439499304589706,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":42.5,"top_score":false},{"model":"dnabert-3","mean":0.367910265754589,"sd":0.004160181399774734,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"dnabert-4","mean":0.5152238797113134,"sd":0.013549069906248424,"seeds":[1,2,4],"note":null,"rank":29.0,"top_score":false},{"model":"dnabert-5","mean":0.5169980394278894,"sd":0.007574662595731279,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"dnabert-6","mean":0.5315418404706131,"sd":0.0008685858361707252,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"dnabert2","mean":0.3746360035340333,"sd":0.01984783049449184,"seeds":[2,3,4],"note":null,"rank":39.0,"top_score":false},{"model":"enformer","mean":0.605764523316179,"sd":0.003828750887687348,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"evo2_1b_base","mean":0.45277101227348865,"sd":0.05844167664097505,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.48067744734470425,"sd":0.005469865044619401,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5426740238812193,"sd":0.0066840980847831305,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.5696475043874905,"sd":0.003682784304116196,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.5451579694111732,"sd":0.006133193603319642,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.6012637262284479,"sd":0.009975957805073578,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3666487930909156,"sd":0.0018274103944493835,"seeds":[2,3,4],"note":null,"rank":41.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7401621520931567,"sd":0.0035460657717123357,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.6947751390575411,"sd":0.00995935163464142,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"grover","mean":0.5013743922493001,"sd":0.002602971922714562,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"hyena_large_1m","mean":0.48733779908320146,"sd":0.0050691325390814805,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4825417741244245,"sd":0.004675902225477776,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.46502820306242393,"sd":0.011533132060300206,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.6181255260165119,"sd":0.003186302791974423,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6835891598524831,"sd":0.011726127438613319,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.6237924100786248,"sd":0.005485495290643122,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5831113974915566,"sd":0.0034994455740805,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.665911846001895,"sd":0.02080481957335301,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5156796411141773,"sd":0.018599173413351246,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.5014385969222696,"sd":0.0086405466350746,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.6238056123516167,"sd":0.005389984329989344,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6611123314726497,"sd":0.005920964177804357,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.6944029759060287,"sd":0.012557398607649876,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5753258972940538,"sd":0.005445204568898337,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"orthrus-4track","mean":0.524383118513767,"sd":0.013508598405842627,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"rinalmo","mean":0.5890103976045832,"sd":0.007645968022528507,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"rnabert","mean":0.36439499304589706,"sd":0.0,"seeds":[1,2,3],"note":null,"rank":42.5,"top_score":false},{"model":"rnaernie","mean":0.5412357627165042,"sd":0.004716164389593583,"seeds":[1,2,3],"note":null,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.49050868780280743,"sd":0.012103689500410365,"seeds":[1,3,4],"note":null,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.6283811134213897,"sd":0.013485926534028267,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false}]},{"id":"rna_localization_rnagps_apexseq","benchmark":"ours_multilabel_classification","name":"RNA Localization Prediction","modality":"RNA","metric":"ap_macro","protocol_variant":"strict_v5_c80","results":[{"model":"alphagenome","mean":0.36765799531248783,"sd":0.009771847059721018,"seeds":[1,2,3],"note":null,"rank":4.0,"top_score":false},{"model":"baseline_cnn","mean":0.26918597199312855,"sd":0.01727465872713844,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"baseline_lstm","mean":0.2994596627855484,"sd":0.003634376204913469,"seeds":[1,2,3],"note":58,"rank":24.0,"top_score":false},{"model":"borzoi","mean":0.2784197337797184,"sd":0.03232488754197635,"seeds":[1,2,3],"note":null,"rank":32.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.28076004239720825,"sd":0.0057268800918428555,"seeds":[1,2,3],"note":null,"rank":30.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.2665696204802288,"sd":0.02276292514077458,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.20623047358763988,"sd":0.007488172040130728,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"dnabert-3","mean":0.27521576534382364,"sd":0.01033064800253075,"seeds":[1,2,3],"note":null,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.2798959133276107,"sd":0.0011421335122901147,"seeds":[1,2,3],"note":null,"rank":31.0,"top_score":false},{"model":"dnabert-5","mean":0.26052537439813245,"sd":0.004672814604346849,"seeds":[1,2,3],"note":null,"rank":38.0,"top_score":false},{"model":"dnabert-6","mean":0.2758579969743586,"sd":0.013534823362806795,"seeds":[1,2,3],"note":null,"rank":34.0,"top_score":false},{"model":"dnabert2","mean":0.21888283797318267,"sd":0.042902596519353985,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"enformer","mean":0.22342188323289863,"sd":0.026538218684400282,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"evo2_1b_base","mean":0.2393281331813134,"sd":0.010135938639309955,"seeds":[1,2,3],"note":86,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.3070295234594699,"sd":0.010416915232250133,"seeds":[1,2,3],"note":null,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.3247304683447671,"sd":0.01348485688728809,"seeds":[1,2,3],"note":null,"rank":12.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3091902166550082,"sd":0.013749328286023576,"seeds":[1,2,3],"note":null,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.2916913429632498,"sd":0.016875390231024138,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3247152470567769,"sd":0.019550693091550678,"seeds":[1,2,3],"note":null,"rank":13.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.2974990598151837,"sd":0.0030601031877793157,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.3939858495068373,"sd":0.013234232600607788,"seeds":[1,2,3],"note":null,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.3089826780913054,"sd":0.011773271981478572,"seeds":[1,2,3],"note":null,"rank":18.0,"top_score":false},{"model":"grover","mean":0.29527428717111226,"sd":0.014423830125227481,"seeds":[1,2,3],"note":null,"rank":26.0,"top_score":false},{"model":"hyena_large_1m","mean":0.30451690621582916,"sd":0.011818148019057055,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.3215299490477889,"sd":0.01093085315049624,"seeds":[1,2,3],"note":87,"rank":15.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.2768237133241759,"sd":0.011448809035310486,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.37334859565050044,"sd":0.015073701020847267,"seeds":[1,2,3],"note":null,"rank":2.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.346981788335251,"sd":0.01370492308908782,"seeds":[1,2,3],"note":null,"rank":6.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.34407526933182914,"sd":0.005552555884188833,"seeds":[1,2,3],"note":null,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.3306004404502294,"sd":0.008950355035330954,"seeds":[1,2,3],"note":null,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.36804069388882876,"sd":0.0050064340027367125,"seeds":[1,2,3],"note":null,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.3076294579933455,"sd":0.0035997300894599583,"seeds":[1,2,3],"note":null,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3318855871568692,"sd":0.011565680475518175,"seeds":[1,2,3],"note":null,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.3236110209550784,"sd":0.004290612277633773,"seeds":[1,2,3],"note":null,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.35043966718091335,"sd":0.015220607418989727,"seeds":[1,2,3],"note":null,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.3324186827383603,"sd":0.022921987284757882,"seeds":[1,2,3],"note":null,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.3292236200902672,"sd":0.006249163899741891,"seeds":[1,2,3],"note":null,"rank":11.0,"top_score":false},{"model":"orthrus-4track","mean":0.30871043917102703,"sd":0.004360180661768241,"seeds":[1,2,3],"note":null,"rank":19.0,"top_score":false},{"model":"rinalmo","mean":0.31288368720568227,"sd":0.0100436605703657,"seeds":[1,2,3],"note":null,"rank":16.0,"top_score":false},{"model":"rnabert","mean":0.20110690178415155,"sd":0.01580597218599874,"seeds":[1,2,3],"note":null,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.3047942285441793,"sd":0.014836513792968633,"seeds":[1,2,3],"note":null,"rank":22.0,"top_score":false},{"model":"rnafm","mean":0.2918599340390262,"sd":0.0038464900149943926,"seeds":[1,2,3],"note":null,"rank":27.0,"top_score":false},{"model":"splicebert","mean":0.2818156912806993,"sd":0.011098269476660011,"seeds":[1,2,3],"note":null,"rank":29.0,"top_score":false}]},{"id":"cpg_methylation_regression","benchmark":"ours_regression","name":"CpG Methylation Prediction (Regression)","modality":"DNA","metric":"spearman","protocol_variant":"full_range_continuous_v1","results":[{"model":"alphagenome","mean":0.4326685353146837,"sd":0.003868341388427725,"seeds":[1,2,3],"note":88,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.3542607451805349,"sd":0.0004827761752538647,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"baseline_lstm","mean":0.3335065717118729,"sd":0.0030897267028741084,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"borzoi","mean":0.3947507938359464,"sd":0.00427751680056207,"seeds":[1,2,3],"note":88,"rank":20.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4034878152793903,"sd":0.0012540488109487984,"seeds":[1,2,3],"note":88,"rank":13.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.40369270606114666,"sd":0.00297198126313449,"seeds":[1,2,3],"note":88,"rank":12.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.352296519044484,"sd":0.007727823643057389,"seeds":[1,2,3],"note":88,"rank":35.0,"top_score":false},{"model":"dnabert-3","mean":0.408547997084278,"sd":0.006734020300659826,"seeds":[1,2,3],"note":88,"rank":10.0,"top_score":false},{"model":"dnabert-4","mean":0.41888255995746315,"sd":0.0046259772595836236,"seeds":[1,2,3],"note":88,"rank":3.0,"top_score":false},{"model":"dnabert-5","mean":0.4099102042557591,"sd":0.002250378590162309,"seeds":[1,2,3],"note":88,"rank":9.0,"top_score":false},{"model":"dnabert-6","mean":0.41562606165920185,"sd":0.005329986106448064,"seeds":[1,2,3],"note":88,"rank":5.0,"top_score":false},{"model":"dnabert2","mean":0.35347399740800717,"sd":0.003950698563421086,"seeds":[1,2,3],"note":88,"rank":34.0,"top_score":false},{"model":"enformer","mean":0.3653154579749397,"sd":0.01571116739601816,"seeds":[1,2,3],"note":88,"rank":32.0,"top_score":false},{"model":"evo2_1b_base","mean":0.38073473593504126,"sd":0.00624355316440038,"seeds":[1,2,3],"note":88,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.34714984727682346,"sd":0.0031419885212696885,"seeds":[1,2,3],"note":88,"rank":38.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.3809020607098043,"sd":0.005593880975177689,"seeds":[1,2,3],"note":88,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3485410192543915,"sd":0.0041568282658904585,"seeds":[1,2,3],"note":88,"rank":37.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.34479026448537825,"sd":0.0050515339933652524,"seeds":[1,2,3],"note":88,"rank":39.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.3948586715768008,"sd":0.008439953298047313,"seeds":[1,2,3],"note":88,"rank":19.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.3852787395287259,"sd":0.01167419717541307,"seeds":[1,2,3],"note":88,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.376061983024679,"sd":0.03608506575125668,"seeds":[1,2,3],"note":88,"rank":30.0,"top_score":false},{"model":"phylogpn","mean":0.35074363884940446,"sd":0.002501861283520715,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"grover","mean":0.38812120416726087,"sd":0.008759922754918485,"seeds":[1,2,3],"note":88,"rank":24.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3702230087373592,"sd":0.020636003986235423,"seeds":[1,2,3],"note":88,"rank":31.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.38806955668674886,"sd":0.00270709299018096,"seeds":[1,2,3],"note":88,"rank":25.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.28734199947672795,"sd":0.0029296301416821117,"seeds":[1,2,3],"note":88,"rank":43.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.38823110767581354,"sd":0.0019489672389322598,"seeds":[1,2,3],"note":88,"rank":23.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.4104535130779168,"sd":0.01012998744290282,"seeds":[1,2,3],"note":88,"rank":8.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.3938873820901224,"sd":0.0005601480984664685,"seeds":[1,2,3],"note":88,"rank":21.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.4154736106233995,"sd":0.0070892082219051805,"seeds":[1,2,3],"note":88,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.4145418955840571,"sd":0.0019755411073473107,"seeds":[1,2,3],"note":88,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.38761840638772455,"sd":0.000983558954407964,"seeds":[1,2,3],"note":88,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.3883651610383018,"sd":0.005672575096043638,"seeds":[1,2,3],"note":88,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.40334061202647964,"sd":0.0031171144355726073,"seeds":[1,2,3],"note":88,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.4078385857798539,"sd":0.006418984130720477,"seeds":[1,2,3],"note":88,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.41692775384319886,"sd":0.008120467027111996,"seeds":[1,2,3],"note":88,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.40252409830970204,"sd":0.0042013793050624405,"seeds":[1,2,3],"note":88,"rank":15.0,"top_score":false},{"model":"orthrus-4track","mean":0.29940285595646865,"sd":0.0032303461754311243,"seeds":[1,2,3],"note":88,"rank":42.0,"top_score":false},{"model":"rinalmo","mean":0.3977907378812525,"sd":0.009276804033449649,"seeds":[1,2,3],"note":88,"rank":18.0,"top_score":false},{"model":"rnabert","mean":0.30941829570593443,"sd":0.003340721117556241,"seeds":[1,2,3],"note":88,"rank":41.0,"top_score":false},{"model":"rnaernie","mean":0.4024085076728028,"sd":0.0048709146778715025,"seeds":[1,2,3],"note":88,"rank":16.0,"top_score":false},{"model":"rnafm","mean":0.40031070078745856,"sd":0.0026107914382284057,"seeds":[1,2,3],"note":88,"rank":17.0,"top_score":false},{"model":"splicebert","mean":0.4226624833780697,"sd":0.005075797609356061,"seeds":[1,2,3],"note":88,"rank":2.0,"top_score":false}]},{"id":"histone_annotation_regression","benchmark":"ours_regression","name":"Epigenetic Marks Prediction (Regression)","modality":"DNA","metric":"spearman","protocol_variant":"paired_continuous_log1p_v1","results":[{"model":"alphagenome","mean":0.6246877569751678,"sd":0.0012001051350343148,"seeds":[1,2,3],"note":88,"rank":1.0,"top_score":true},{"model":"baseline_cnn","mean":0.3697219399301385,"sd":0.001890987574870395,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.44688881257454344,"sd":0.003185370406810891,"seeds":[1,2,3],"note":null,"rank":33.0,"top_score":false},{"model":"borzoi","mean":0.6007465964790393,"sd":0.0020533780306283165,"seeds":[1,2,3],"note":88,"rank":2.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.4942088125887139,"sd":0.003887675580850727,"seeds":[1,2,3],"note":88,"rank":19.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5166345310931515,"sd":0.01132781759210395,"seeds":[1,2,3],"note":89,"rank":10.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.40088701769017054,"sd":0.0035558495134945076,"seeds":[1,2,3],"note":88,"rank":38.0,"top_score":false},{"model":"dnabert-3","mean":0.3997347284602766,"sd":0.02193841655292811,"seeds":[2,3,4],"note":90,"rank":39.0,"top_score":false},{"model":"dnabert-4","mean":0.3945166395945305,"sd":0.002594160793631829,"seeds":[1,2,3],"note":88,"rank":40.0,"top_score":false},{"model":"dnabert-5","mean":0.40596465686202304,"sd":0.007093472639048829,"seeds":[1,2,3],"note":88,"rank":37.0,"top_score":false},{"model":"dnabert-6","mean":0.4084769511772152,"sd":0.006575353318003433,"seeds":[1,2,3],"note":88,"rank":36.0,"top_score":false},{"model":"dnabert2","mean":0.5015817916973611,"sd":0.005831869323938712,"seeds":[1,2,3],"note":88,"rank":16.0,"top_score":false},{"model":"enformer","mean":0.5902091459330076,"sd":0.002512544920857912,"seeds":[1,2,3],"note":88,"rank":3.0,"top_score":false},{"model":"evo2_1b_base","mean":0.4672971803125489,"sd":0.007045498388317103,"seeds":[1,2,3],"note":91,"rank":28.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.45743992067485256,"sd":0.005746989191971393,"seeds":[1,2,3],"note":88,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5033455398416968,"sd":0.0026856507700723066,"seeds":[1,2,3],"note":88,"rank":15.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.4546454267836146,"sd":0.004379752917422962,"seeds":[1,2,3],"note":88,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.44986192943958614,"sd":0.008433553040306602,"seeds":[1,2,3],"note":88,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5162084414293764,"sd":0.0163650697373902,"seeds":[1,2,3],"note":88,"rank":11.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.5081740584286494,"sd":0.008535032558284786,"seeds":[1,2,3],"note":88,"rank":14.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5204529298032131,"sd":0.003460907490636883,"seeds":[1,2,3],"note":88,"rank":9.0,"top_score":false},{"model":"phylogpn","mean":0.4859019860821856,"sd":0.0024048020663071383,"seeds":[1,2,3],"note":null,"rank":25.0,"top_score":false},{"model":"grover","mean":0.4919269491853706,"sd":0.00436933857561894,"seeds":[1,2,3],"note":88,"rank":20.0,"top_score":false},{"model":"hyena_large_1m","mean":0.4291147905758994,"sd":0.0014779032702934175,"seeds":[1,2,3],"note":88,"rank":35.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4903778614448358,"sd":0.0068668652510504155,"seeds":[1,2,3],"note":88,"rank":22.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.3941966269635246,"sd":0.003081035499195665,"seeds":[1,2,3],"note":88,"rank":41.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.500264822226575,"sd":0.0028062943340597286,"seeds":[1,2,3],"note":88,"rank":17.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.5429285665924101,"sd":0.002783564545983183,"seeds":[1,2,3],"note":88,"rank":5.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.468118548510632,"sd":0.004176548430163534,"seeds":[1,2,3],"note":88,"rank":27.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.49926630943053085,"sd":0.00036473234593755295,"seeds":[1,2,3],"note":88,"rank":18.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5390105919726518,"sd":0.00585046989714302,"seeds":[1,2,3],"note":88,"rank":6.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.45926985462521136,"sd":0.009819425341100854,"seeds":[1,2,3],"note":88,"rank":29.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.47011148530609737,"sd":0.005081204266180569,"seeds":[1,2,3],"note":88,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5085082045582248,"sd":0.0019427380362589167,"seeds":[1,2,3],"note":88,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.5284798135358457,"sd":0.003457776341246576,"seeds":[1,2,3],"note":88,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5435112230860341,"sd":0.0032641592486936486,"seeds":[1,2,3],"note":88,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.49089315121241245,"sd":0.0050747318987573245,"seeds":[1,2,3],"note":88,"rank":21.0,"top_score":false},{"model":"orthrus-4track","mean":0.44088676857514686,"sd":0.005033396459951358,"seeds":[1,2,3],"note":88,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.511945426282583,"sd":0.007289497449687783,"seeds":[1,2,3],"note":88,"rank":12.0,"top_score":false},{"model":"rnabert","mean":0.2136066202602926,"sd":0.0073106182134318265,"seeds":[1,2,3],"note":88,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.4877441111702649,"sd":0.012943816429601155,"seeds":[1,2,3],"note":88,"rank":24.0,"top_score":false},{"model":"rnafm","mean":0.48893073576859464,"sd":0.008480066457654737,"seeds":[1,2,3],"note":88,"rank":23.0,"top_score":false},{"model":"splicebert","mean":0.5205734815765626,"sd":0.002715264848165245,"seeds":[1,2,3],"note":88,"rank":8.0,"top_score":false}]},{"id":"mrna_half_life_saluki_human","benchmark":"ours_regression","name":"mRNA Stability Prediction","modality":"RNA","metric":"spearman","protocol_variant":"author_f1_c1_sequence_only_v1","results":[{"model":"alphagenome","mean":0.6690566844174173,"sd":0.004507929012781528,"seeds":[1,2,3],"note":92,"rank":3.0,"top_score":false},{"model":"baseline_cnn","mean":0.2527985629291719,"sd":0.027683238892651667,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"baseline_lstm","mean":0.3116939124954342,"sd":0.02950730628963851,"seeds":[1,2,3],"note":null,"rank":36.0,"top_score":false},{"model":"borzoi","mean":0.565519693298464,"sd":0.005820220402628264,"seeds":[1,2,3],"note":93,"rank":11.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.5480428095432686,"sd":0.01641073195371316,"seeds":[1,2,3],"note":93,"rank":14.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.5299574821357814,"sd":0.02215456773815616,"seeds":[1,2,3],"note":93,"rank":17.0,"top_score":false},{"model":"cnn-56m-random-init","mean":-0.02670400232552078,"sd":0.01563762600634917,"seeds":[1,2,3],"note":93,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.2904126400687471,"sd":0.03104277939907015,"seeds":[1,2,3],"note":93,"rank":37.0,"top_score":false},{"model":"dnabert-4","mean":0.28289615119733424,"sd":0.024135330981097776,"seeds":[1,2,3],"note":93,"rank":38.0,"top_score":false},{"model":"dnabert-5","mean":0.2718220840191165,"sd":0.002558974131779299,"seeds":[1,2,3],"note":93,"rank":39.0,"top_score":false},{"model":"dnabert-6","mean":0.2632042512350888,"sd":0.025298880418020045,"seeds":[1,2,3],"note":93,"rank":40.0,"top_score":false},{"model":"dnabert2","mean":0.5090451591020927,"sd":0.013034678574859772,"seeds":[1,2,3],"note":93,"rank":19.0,"top_score":false},{"model":"enformer","mean":0.4986654872272022,"sd":0.022206912547801896,"seeds":[1,2,3],"note":93,"rank":22.0,"top_score":false},{"model":"evo2_1b_base","mean":0.5517066581719792,"sd":0.01123124148783467,"seeds":[1,2,3],"note":94,"rank":13.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.4875220709207157,"sd":0.01681175680471369,"seeds":[1,2,3],"note":93,"rank":24.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.5033810696147951,"sd":0.011241872717023208,"seeds":[1,2,3],"note":93,"rank":21.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.45293797511016876,"sd":0.02445518804813617,"seeds":[1,2,3],"note":93,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.46182341611213734,"sd":0.016136438049042728,"seeds":[1,2,3],"note":93,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.5321849104801946,"sd":0.023902614338530876,"seeds":[1,2,3],"note":93,"rank":16.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.4676153745809729,"sd":0.03292220420044788,"seeds":[1,2,3],"note":93,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.7147385338862189,"sd":0.005606138168897932,"seeds":[1,2,3],"note":93,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.4897649376195728,"sd":0.012288506420669079,"seeds":[1,2,3],"note":null,"rank":23.0,"top_score":false},{"model":"grover","mean":0.45077852273014535,"sd":0.015465147670577686,"seeds":[1,2,3],"note":93,"rank":31.0,"top_score":false},{"model":"hyena_large_1m","mean":0.47037444252025695,"sd":0.01844135036504559,"seeds":[1,2,3],"note":95,"rank":25.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4661718177025293,"sd":0.019604109800883913,"seeds":[1,2,3],"note":95,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.357835837239,"sd":0.011386808087986122,"seeds":[1,2,3],"note":93,"rank":35.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.636126542746141,"sd":0.002449532186230108,"seeds":[1,2,3],"note":93,"rank":6.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6710756314301543,"sd":0.012131269204000641,"seeds":[1,2,3],"note":93,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.5976456364144921,"sd":0.01691057685159873,"seeds":[1,2,3],"note":93,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.561847264384366,"sd":0.0029323863741019156,"seeds":[1,2,3],"note":93,"rank":12.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.6393528000110805,"sd":0.005900405068003554,"seeds":[1,2,3],"note":93,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.5065448820917584,"sd":0.009581373203439863,"seeds":[1,2,3],"note":93,"rank":20.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.470293766026754,"sd":0.029803320699468706,"seeds":[1,2,3],"note":93,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5702983871704254,"sd":0.002151740900148424,"seeds":[1,2,3],"note":93,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.6211373823765693,"sd":0.005817124402665871,"seeds":[1,2,3],"note":93,"rank":7.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.639498481474119,"sd":0.0021152929101545034,"seeds":[1,2,3],"note":93,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.5474690773598135,"sd":0.011244398946136665,"seeds":[1,2,3],"note":93,"rank":15.0,"top_score":false},{"model":"orthrus-4track","mean":0.3669879676915541,"sd":0.013447654112352653,"seeds":[1,2,3],"note":93,"rank":34.0,"top_score":false},{"model":"rinalmo","mean":0.6175650719300929,"sd":0.013798976431508202,"seeds":[1,2,3],"note":93,"rank":8.0,"top_score":false},{"model":"rnabert","mean":0.0634631483331922,"sd":0.02316560314896833,"seeds":[1,2,3],"note":93,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.39493053124459093,"sd":0.017951379856445217,"seeds":[1,2,4],"note":96,"rank":33.0,"top_score":false},{"model":"rnafm","mean":0.4196165257594456,"sd":0.0010291207682330804,"seeds":[1,2,3],"note":93,"rank":32.0,"top_score":false},{"model":"splicebert","mean":0.5109543143775725,"sd":0.0032079305392707287,"seeds":[1,2,3],"note":93,"rank":18.0,"top_score":false}]},{"id":"translation_efficiency_ribonn_human_mean","benchmark":"ours_regression","name":"Translation Efficiency Prediction","modality":"RNA","metric":"spearman","protocol_variant":"author_outer0_inner0_sequence_only_v1","results":[{"model":"alphagenome","mean":0.7721392365042641,"sd":0.001589763518395336,"seeds":[1,2,3],"note":97,"rank":6.0,"top_score":false},{"model":"baseline_cnn","mean":0.4766290178127255,"sd":0.018144262383599143,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"baseline_lstm","mean":0.3247582525128375,"sd":0.014956237077992594,"seeds":[1,2,3],"note":null,"rank":41.0,"top_score":false},{"model":"borzoi","mean":0.705982636053199,"sd":0.0008528178255160579,"seeds":[1,2,3],"note":97,"rank":15.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.7190364823435484,"sd":0.016799812583125386,"seeds":[1,2,3],"note":97,"rank":12.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.7025647533047398,"sd":0.003052963842304193,"seeds":[1,2,3],"note":97,"rank":16.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.005302098265309763,"sd":0.01918580131355151,"seeds":[1,2,3],"note":97,"rank":43.0,"top_score":false},{"model":"dnabert-3","mean":0.5103738596665846,"sd":0.02117756083197484,"seeds":[1,2,3],"note":97,"rank":35.0,"top_score":false},{"model":"dnabert-4","mean":0.5104822441065472,"sd":0.012184068806905677,"seeds":[1,2,3],"note":97,"rank":34.0,"top_score":false},{"model":"dnabert-5","mean":0.49017259235352495,"sd":0.0034887364179259665,"seeds":[1,2,3],"note":97,"rank":37.0,"top_score":false},{"model":"dnabert-6","mean":0.48859916612125553,"sd":0.025152520368382683,"seeds":[1,2,3],"note":97,"rank":38.0,"top_score":false},{"model":"dnabert2","mean":0.6703705467946677,"sd":0.010322003568843373,"seeds":[1,2,3],"note":97,"rank":20.0,"top_score":false},{"model":"enformer","mean":0.63408553990503,"sd":0.013698071222425133,"seeds":[1,2,3],"note":97,"rank":27.0,"top_score":false},{"model":"evo2_1b_base","mean":0.7402710078688508,"sd":0.014315323091039103,"seeds":[1,2,3],"note":94,"rank":9.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.6111019564381054,"sd":0.010043054526252468,"seeds":[1,2,3],"note":97,"rank":32.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.6762193011495004,"sd":0.0026382578898611567,"seeds":[1,2,3],"note":97,"rank":18.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.6260256644532558,"sd":0.01161434668985807,"seeds":[1,2,3],"note":97,"rank":30.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.6207468693466834,"sd":0.007661986514209977,"seeds":[1,2,3],"note":97,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.7060718440600182,"sd":0.019392919701526446,"seeds":[1,2,3],"note":97,"rank":14.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.6568065018984623,"sd":0.00512686202181271,"seeds":[1,2,3],"note":97,"rank":21.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.8093621148268071,"sd":0.0020535457448540803,"seeds":[1,2,3],"note":97,"rank":1.0,"top_score":true},{"model":"phylogpn","mean":0.632719669987694,"sd":0.014246382241646389,"seeds":[1,2,3],"note":null,"rank":28.0,"top_score":false},{"model":"grover","mean":0.6314826661802787,"sd":0.004902848354630267,"seeds":[1,2,3],"note":97,"rank":29.0,"top_score":false},{"model":"hyena_large_1m","mean":0.653868617140069,"sd":0.008246799765954124,"seeds":[1,2,3],"note":98,"rank":23.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.6478153181463409,"sd":0.00723822621646437,"seeds":[1,2,3],"note":98,"rank":24.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.5055175162191989,"sd":0.010023905135306776,"seeds":[1,2,3],"note":97,"rank":36.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.7580175655542336,"sd":0.0039067733605185374,"seeds":[1,2,3],"note":97,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.7903946706130078,"sd":0.014143207274691982,"seeds":[1,2,3],"note":97,"rank":2.0,"top_score":false},{"model":"ntv3-8m-pre","mean":0.7415226569116384,"sd":0.007603807807069354,"seeds":[1,2,3],"note":97,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.736184115662198,"sd":0.0058051011891126145,"seeds":[1,2,3],"note":97,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.7822120234542564,"sd":0.00845843745001949,"seeds":[1,2,3],"note":97,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.6423590313030468,"sd":0.011206957569647636,"seeds":[1,2,3],"note":97,"rank":26.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.653960759825341,"sd":0.00363176707623867,"seeds":[1,2,3],"note":97,"rank":22.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.7075135416091161,"sd":0.007861938849608997,"seeds":[1,2,3],"note":97,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.7317577868888333,"sd":0.009580165629825058,"seeds":[1,2,3],"note":97,"rank":11.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.7793910721404584,"sd":0.005990309697580716,"seeds":[1,2,3],"note":97,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.6876039175736947,"sd":0.0076250877472927355,"seeds":[1,2,3],"note":97,"rank":17.0,"top_score":false},{"model":"orthrus-4track","mean":0.4861962427527352,"sd":0.021942951823842587,"seeds":[1,2,3],"note":97,"rank":39.0,"top_score":false},{"model":"rinalmo","mean":0.7792528282043115,"sd":0.0020233686544959173,"seeds":[1,2,3],"note":97,"rank":5.0,"top_score":false},{"model":"rnabert","mean":0.25611127771694514,"sd":0.00572029819080204,"seeds":[1,2,3],"note":97,"rank":42.0,"top_score":false},{"model":"rnaernie","mean":0.6437895914669144,"sd":0.01898898783698965,"seeds":[1,2,3],"note":97,"rank":25.0,"top_score":false},{"model":"rnafm","mean":0.5757880042584219,"sd":0.014433002654055534,"seeds":[1,2,3],"note":97,"rank":33.0,"top_score":false},{"model":"splicebert","mean":0.6755696040630308,"sd":0.013888730752994126,"seeds":[1,2,3],"note":97,"rank":19.0,"top_score":false}]},{"id":"protein_localization_deeploc2","benchmark":"ours_multilabel_classification","name":"Protein Subcellular Localization (DeepLoc2)","modality":"Protein","metric":"ap_macro","protocol_variant":"paired_cds_le4096_v2_20260920","results":[{"model":"alphagenome","mean":0.4521990081599492,"sd":0.010651527315135963,"seeds":[1,2,3],"note":99,"rank":15.0,"top_score":false},{"model":"baseline_cnn","mean":0.258489706401784,"sd":0.009873625116567102,"seeds":[1,2,3],"note":null,"rank":42.0,"top_score":false},{"model":"baseline_lstm","mean":0.350847718996112,"sd":0.01638260873317396,"seeds":[1,2,3],"note":null,"rank":37.0,"top_score":false},{"model":"borzoi","mean":0.3652558784989546,"sd":0.12612137052203876,"seeds":[2,3,4],"note":100,"rank":36.0,"top_score":false},{"model":"caduceus_ph_seqlen_131k_d_model_256","mean":0.38744797926371227,"sd":0.04756273201367699,"seeds":[1,2,3],"note":101,"rank":34.0,"top_score":false},{"model":"caduceus_ps_seqlen_131k_d_model_256","mean":0.4185623551054755,"sd":0.01957992794005155,"seeds":[1,2,3],"note":102,"rank":22.0,"top_score":false},{"model":"cnn-56m-random-init","mean":0.4318277523991922,"sd":0.003850971626675365,"seeds":[1,2,3],"note":103,"rank":20.0,"top_score":false},{"model":"dnabert-3","mean":0.4341403605348635,"sd":0.0028310879241473623,"seeds":[1,2,3],"note":99,"rank":18.0,"top_score":false},{"model":"dnabert-4","mean":0.43343714176620224,"sd":0.015718341083575872,"seeds":[1,2,3],"note":99,"rank":19.0,"top_score":false},{"model":"dnabert-5","mean":0.4179817234418101,"sd":0.009929965651747592,"seeds":[1,2,3],"note":99,"rank":23.0,"top_score":false},{"model":"dnabert-6","mean":0.3910047209473349,"sd":0.01337941798552511,"seeds":[1,2,4],"note":104,"rank":32.0,"top_score":false},{"model":"dnabert2","mean":0.4099970263856653,"sd":0.0027931643090153445,"seeds":[1,2,3],"note":99,"rank":26.0,"top_score":false},{"model":"enformer","mean":0.3295552837900726,"sd":0.003285657119145364,"seeds":[1,2,3],"note":99,"rank":38.0,"top_score":false},{"model":"evo2_1b_base","mean":0.4403327976125856,"sd":0.03281107342974725,"seeds":[1,2,3],"note":105,"rank":17.0,"top_score":false},{"model":"gena-lm-bert-base","mean":0.3891750415965676,"sd":0.005308623976971917,"seeds":[1,2,3],"note":99,"rank":33.0,"top_score":false},{"model":"gena-lm-bert-base-lastln-t2t","mean":0.41290249458911066,"sd":0.01200119250416489,"seeds":[1,2,3],"note":99,"rank":25.0,"top_score":false},{"model":"gena-lm-bert-base-t2t","mean":0.3966714835711255,"sd":0.0007226934747977021,"seeds":[1,2,3],"note":99,"rank":29.0,"top_score":false},{"model":"gena-lm-bert-base-t2t-multi","mean":0.3951813422206648,"sd":0.006771905634844181,"seeds":[1,2,3],"note":99,"rank":31.0,"top_score":false},{"model":"gena-lm-bert-large-t2t","mean":0.43092041993327135,"sd":0.001918092362148185,"seeds":[1,2,3],"note":99,"rank":21.0,"top_score":false},{"model":"gena-lm-bigbird-base-t2t","mean":0.40546207284214064,"sd":0.00801648967373668,"seeds":[1,2,3],"note":99,"rank":27.0,"top_score":false},{"model":"generator_eukaryote_1_2b","mean":0.5886246592048651,"sd":0.007828593913139122,"seeds":[1,2,3],"note":99,"rank":2.0,"top_score":false},{"model":"phylogpn","mean":0.2997833430760028,"sd":0.00708487285872378,"seeds":[1,2,3],"note":null,"rank":40.0,"top_score":false},{"model":"grover","mean":0.3796756873119619,"sd":0.004038053911692775,"seeds":[1,2,3],"note":99,"rank":35.0,"top_score":false},{"model":"hyena_large_1m","mean":0.3957431410142804,"sd":0.004238602790913479,"seeds":[1,2,3],"note":102,"rank":30.0,"top_score":false},{"model":"hyena_medium_160k","mean":0.4029586147767331,"sd":0.005625646469933534,"seeds":[1,2,3],"note":106,"rank":28.0,"top_score":false},{"model":"ntv2-50m-random-init","mean":0.3118312014722211,"sd":0.008560758723679193,"seeds":[1,2,3],"note":107,"rank":39.0,"top_score":false},{"model":"ntv3-100m-pre","mean":0.5239208483485521,"sd":0.009102203572319993,"seeds":[1,2,3],"note":102,"rank":7.0,"top_score":false},{"model":"ntv3-650m-pre","mean":0.6037739601953084,"sd":0.022517319551920677,"seeds":[1,2,3],"note":102,"rank":1.0,"top_score":true},{"model":"ntv3-8m-pre","mean":0.5207506237304083,"sd":0.00328182318788414,"seeds":[1,2,3],"note":102,"rank":9.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-1000g","mean":0.5096746777316223,"sd":0.004781110310656271,"seeds":[1,2,3],"note":99,"rank":10.0,"top_score":false},{"model":"nucleotide-transformer-2.5b-multi-species","mean":0.5572569706466555,"sd":0.008909590929761839,"seeds":[1,2,3],"note":99,"rank":5.0,"top_score":false},{"model":"nucleotide-transformer-500m-1000g","mean":0.47826729722272293,"sd":0.011516085189846214,"seeds":[1,2,3],"note":99,"rank":13.0,"top_score":false},{"model":"nucleotide-transformer-500m-human-ref","mean":0.46861904815417843,"sd":0.007024391099254258,"seeds":[1,2,3],"note":99,"rank":14.0,"top_score":false},{"model":"nucleotide-transformer-v2-100m-multi-species","mean":0.5220143395653801,"sd":0.00276987651949306,"seeds":[1,2,3],"note":99,"rank":8.0,"top_score":false},{"model":"nucleotide-transformer-v2-250m-multi-species","mean":0.560705303049892,"sd":0.015349378888993495,"seeds":[1,2,3],"note":99,"rank":4.0,"top_score":false},{"model":"nucleotide-transformer-v2-500m-multi-species","mean":0.5744950584328969,"sd":0.007848536130755875,"seeds":[1,2,3],"note":99,"rank":3.0,"top_score":false},{"model":"nucleotide-transformer-v2-50m-multi-species","mean":0.4876729827372856,"sd":0.011999134202680034,"seeds":[1,2,3],"note":99,"rank":12.0,"top_score":false},{"model":"orthrus-4track","mean":0.28243314151426574,"sd":0.005956488810493253,"seeds":[1,2,3],"note":99,"rank":41.0,"top_score":false},{"model":"rinalmo","mean":0.5441495207014928,"sd":0.006332464503191482,"seeds":[1,2,3],"note":99,"rank":6.0,"top_score":false},{"model":"rnabert","mean":0.24038580554770456,"sd":0.10083660684714812,"seeds":[2,3,4],"note":104,"rank":43.0,"top_score":false},{"model":"rnaernie","mean":0.4432056535870864,"sd":0.0024474634377417035,"seeds":[1,2,3],"note":99,"rank":16.0,"top_score":false},{"model":"rnafm","mean":0.41759716736015956,"sd":0.020228673447897236,"seeds":[1,2,3],"note":99,"rank":24.0,"top_score":false},{"model":"splicebert","mean":0.4982560720513915,"sd":0.006372780609172178,"seeds":[1,2,3],"note":99,"rank":11.0,"top_score":false}]}],"notes":["Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","Full source sequences; each model retains its own input cap. LR inherited without a new search; fixed seeds 1/2/3.","Full source sequences; each model retains its own input cap. LR inherited without a new search; fixed seeds 1/2/3.; Evo2 inherited numerical-update exception: {\"adaptation\": \"full_backbone_finetune_allow_numerically_inactive_last_layer_v1\", \"exception_used\": true, \"strict_gate_passed\": false, \"update_audit_scope\": \"first_optimizer_step\", \"updated_layer_count\": 24}","Full source sequences; model input cap retained; LR 2e-5 inherited without a new search. User-selected actual seeds 1,4,5 post hoc by highest test macro-AUPRC among seeds 1-5; test macro-F1 yields the same subset. All five runs retained. This is test-informed seed selection, not a prespecified three-seed estimate. Future exports must retain actual seeds 1,4,5.","Full source sequences; each model retains its own input cap. LR inherited without a new search; fixed seeds 1/2/3.; Evo2 inherited numerical-update exception: {\"adaptation\": \"full_backbone_finetune_allow_numerically_inactive_last_layer_v1\", \"exception_used\": true, \"strict_gate_passed\": false, \"update_audit_scope\": \"first_optimizer_step\", \"updated_layer_count\": 24}; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","Full source sequences; each model retains its own input cap. LR inherited without a new search; fixed seeds 1/2/3.; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained","accepted_after_test_review;mixed_fp16_bf16;seed1_6_fp16_seed7_bf16;posthoc_precision_and_validity_selection;all_valid_distinct_seed_results_retained;no_test_score_or_std_subset_ranking;std_not_pure_seed_variation;invalid_attempts_retained","Explicit user-selected actual seeds after inspection of test results on 2026-10-05. This is a post-hoc subset, not a prespecified three-seed stability estimate. All candidate runs and previous accepted results are retained; LR is unchanged.","technical_failure_seed_replacement;accepted_after_test_review;retain_all_valid_completed_seeds;seed2_3_5_6_8_9_nonfinite_predictions;seed7_late_training_nan_but_selected_checkpoint_predictions_finite;fp16;filter_lr_fixed_6e-4","accepted_after_test_review;first_three_valid_seed_numbers;no_test_score_or_std_subset_ranking;fp16;invalid_attempts_retained","accepted_after_test_review;technical_failure_seed_replacement;seed7_additional_run_not_stopped;seed2_4_nonfinite;seed5_user_cancelled","supplementary_lr_grid;filter_lr_tied_protocol;accepted_after_test_review;all_planned_seeds_retained","Evo2 inherited numerical-update exception: full_backbone_finetune_allow_numerically_inactive_last_layer_v1","accepted_after_test_review;posthoc_user_lr_override;all_planned_seeds_retained;low_absolute_performance;discrete_predictions;seed3_1066_of_1080_equal;fp16","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change","accepted_after_test_review;all_valid_seeds_at_selected_lr_retained;posthoc_user_lr_override;fixed_seed123_retained;absolute_mean_spearman_below_0.1;over_90_percent_predictions_equal_in_some_seed","user_confirmed_after_test_review;no_test_score_seed_ranking;failed_attempts_retained;posthoc_user_lr_override;near_constant_predictions_3_5_2_unique_values;not_evidence_of_predictive_discrimination","Posthoc selection of highest test Spearman among five seeds; near-constant predictions; optimistic selected-seed estimate","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;absolute_mean_spearman_below_0.1;nonpositive_seed_spearman;over_90_percent_predictions_equal_in_some_seed","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;std_above_0.1","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;absolute_mean_spearman_below_0.1;nonpositive_seed_spearman","posthoc_top3_test_selection;low_test_spearman;not_unbiased_fixed_seed_repetitions","low_performance;near_constant_predictions;posthoc_user_lr_override","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;valid_seed11_excluded_for_severe_prediction_degeneration;absolute_mean_spearman_below_0.1","Weak correlation; user chose LR after inspecting test predictions; no claim of validation-only LR selection","accepted_after_test_review;all_valid_seeds_at_selected_lr_retained;posthoc_user_lr_override;fixed_seed123_retained;absolute_mean_spearman_below_0.1","accepted_after_test_review;posthoc_user_lr_override;user_fixed123_subset;near_zero_performance;two_prediction_values_each;seed3_9244_of_9245_equal;fp16;not_evidence_of_discrimination","Near-constant predictions; LR1e-5 explicitly chosen after viewing test results of exploratory LRs; fixed seeds1/2/3, no seed filtering","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=user_confirmed_lr1e-5;minimum_validation_loss_of_expanded_five_lr_grid;grid_expanded_after_test_review;seed23_pending;only_first_three_validation_epochs_finite;later_training_nan;best_checkpoint_test_finite; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","accepted_after_test_review;posthoc_user_lr_override;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;discrete_predictions;concentrated_predictions;low_absolute_performance;valid_seeds2_and5_unselected_by_test_score","User requested FP32-head reevaluation and highest three test Spearman seeds; posthoc test-informed seed and inference-precision selection; 14 retained checkpoints compared, seeds1/2/3 missing checkpoints and excluded; predictions remain near-constant; all attempts retained","user_confirmed_after_test_review;posthoc_user_lr_override;near_constant_quantized_predictions;not_evidence_of_predictive_discrimination;all_attempts_retained;fixed_seeds_1_2_3","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=user_confirmed_lr1e-5;minimum_validation_loss_of_expanded_five_lr_grid;grid_expanded_after_test_review;seed23_pending","user_confirmed_after_test_review;posthoc_user_lr_override;near_constant_quantized_predictions;not_evidence_of_predictive_discrimination;all_attempts_retained;test_informed_user_seed_selection_1_3_4;finite_negative_seed2_excluded_by_user;constant_seed5_excluded","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=user_requested_1e-5_after_validation_collapse_and_test_review;not_min_val_loss;posthoc_override;seed23_at_new_lr_pending","Near-constant predictions: each seed has only two unique outputs; user confirmed LR6e-4 and finite seeds5/7/11 after reviewing test results; posthoc test-informed LR/seed acceptance, not fixed seed123; all other attempts retained","near_constant_quantized_predictions;user_accepted_after_test_review;posthoc_lr_override;all_three_seeds_included;not_evidence_of_predictive_discrimination","posthoc_user_lr_override;low_test_spearman;all_planned_seeds_retained","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;valid_seeds8_and20_unselected_by_test_score;selected_seed21_severe_prediction_degeneration;discrete_predictions;absolute_mean_spearman_below_0.1;two_value_predictions_in_some_seeds;over_90_percent_predictions_equal_in_some_seed","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=user_selected_lr1e-5_after_test_degradation_review;not_minimum_validation_loss;seed1_nonconstant;seed23_pending","posthoc_user_lr_override;all_planned_seeds_at_accepted_lr_retained","posthoc_user_lr_override;all_planned_seeds_retained;severe_near_constant_predictions;negative_test_spearman","posthoc_top3_test_selection;posthoc_std_screening;not_unbiased_fixed_seed_repetitions","accepted_after_test_review;posthoc_precision_override_fp16_to_bf16;no_mixed_precision_within_selected_triplet;all_attempts_retained;user_fixed_seed678_after_test_review","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=six_lr_seed1_validation_history;new_lr_seed23_pending","User selected actual seeds 4,5,11 post hoc by highest test Spearman among completed FP32-head reevaluations 4,5,11,12 on 2026-10-05. All four runs retained. Test-informed seed selection, not a prespecified three-seed estimate. LR 1e-4 and FP16-backbone/FP32-head inference inherited; predictions remain near-constant. Seeds 1/2/3 lack retained checkpoints and seeds 6-10 were stopped/incomplete. Future exports use fixed actual seeds 4,5,11.","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;discrete_predictions;two_value_predictions_in_some_seeds;over_90_percent_predictions_equal_in_some_seed","accepted_after_test_review;posthoc_user_lr_override;posthoc_top3_test_seed_selection;repeated_seed_search;discrete_predictions;valid_seeds2_4_unselected","Posthoc selection of highest test Spearman among all completed runs; near-constant predictions; optimistic selected-seed estimate","posthoc_top3_test_selection;posthoc_user_lr_override;std_above_0.1;seed3_near_constant","accepted_after_test_review;posthoc_user_lr_override;technical_failure_seed_replacement;all_valid_seeds_at_selected_lr_retained;discrete_predictions","accepted_after_test_review;posthoc_precision_override_fp16_to_bf16;no_mixed_precision_within_selected_triplet;all_attempts_retained;posthoc_top3_test_spearman_seed_selection","supplementary_lr_grid;filter_lr_tied_protocol;accepted_after_test_review;all_planned_seeds_retained;seed3_checkpoint_resume_after_preemption","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;discrete_predictions","User authorized corrected binmask seed 4 admission on 2026-10-06. Retain seeds 2/3/4, the highest three finite test Spearman scores among correct-protocol seeds 1/2/3/4. Test-informed post-hoc seed selection; subset SD is not a prespecified unbiased three-seed stability estimate. LR, processed CSVs and pooling protocol unchanged. All raw runs retained.","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;absolute_mean_spearman_below_0.1;nonpositive_seed_spearman;two_value_predictions_in_some_seeds","Resumed from durable checkpoint after micro-batch increase; effective batch retained; original non-fault-tolerant sampler does not guarantee uninterrupted sample order","BF16/FP8 last-layer numerical-update exception retained; three-candidate validation-loss LR selection; fixed seeds123","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;absolute_mean_spearman_below_0.1","retrospective_validity_filtering;ascending_valid_seed_numbers;no_test_score_or_std_ranking;invalid_attempts_retained;no_precision_change;historical_lr_selection_notes=failed_validation_candidates=2","supplementary_lr_grid;filter_lr_tied_protocol;accepted_after_test_review;all_planned_seeds_retained;seed2_3_checkpoint_resume_after_preemption;fp16","user_confirmed_after_test_review;no_test_score_seed_ranking;failed_attempts_retained;mixed_fp16_bf16;std_not_pure_seed_variation","user_confirmed_after_test_score_review;all_three_finite_seeds;no_score_ranking;invalid_attempts_retained;no_precision_change;not_prospective_fixed_seed123","accepted_after_test_review;first_three_valid_seed_numbers;no_test_score_or_std_subset_ranking;fp16;invalid_attempts_retained;seed3_late_training_nan_but_best_checkpoint_test_valid","technical_failure_seed_replacement;accepted_after_test_review;retain_all_valid_completed_seeds;seed2_4_nonfinite_predictions;seed5_stopped_by_user_after_nonfinite_training;fp16;filter_lr_fixed_6e-4","user_confirmed_after_test_review;all_three_finite_seeds;no_test_score_ranking;mixed_fp16_bf16;seed1_fp16_seed5_6_bf16;seed5_6_resumed_after_preemption;mid_epoch_resume_not_bitwise_uninterrupted;std_not_pure_seed_variation;failed_attempts_retained","posthoc_user_lr_override;all_planned_seeds_at_accepted_lr_retained;technical_failure_seed_replacement;microbatch_changed_from_old_lr","test_score_top3_posthoc_selection;mixed_fp16_bf16;seed1_fp16_seed5_6_bf16;std_not_pure_seed_variation;all_failed_and_unselected_attempts_retained","user_confirmed_after_test_score_review;all_three_finite_seeds;no_score_ranking;invalid_attempts_retained;no_precision_change;not_prospective_fixed_seed123;some_seeds_have_only_2_or_3_distinct_predictions","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;repeated_seed_search_after_test_review;valid_seed6_excluded_for_severe_prediction_degeneration","posthoc_top3_test_selection;posthoc_user_lr_override;low_performance;concentrated_predictions;not_unbiased_fixed_seed_repetitions","user_requested_top3_test_spearman;test_informed_lr_and_seed_selection;best3_of4_finite_seeds;five_seeds_attempted;invalid_and_unselected_attempts_retained;no_precision_change;only_2_or_3_distinct_test_prediction_values;not_prospective_fixed_seed123;historical_lr_selection_notes=user_requested_lr3e4_after_test_review_20260921;seed1_test_spearman_0.12805015338;seed23_pending;not_validation_minimum","accepted_after_test_review;posthoc_user_specified_seed_subset;posthoc_top3_test_seed_selection;valid_seed11_excluded_for_severe_prediction_degeneration","posthoc_user_lr_override;technical_failure_seed_replacement;std_above_0.1;concentrated_predictions; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","accepted_after_test_review;all_valid_seeds_at_selected_lr_retained;technical_failure_seed_replacement;std_above_0.1;over_90_percent_predictions_equal_in_some_seed","user_confirmed_after_test_score_review;coincides_with_first_three_valid_seed_numbers;invalid_attempts_retained;no_precision_change;only_2_or_3_distinct_test_prediction_values;not_prospective_fixed_seed123","accepted_after_test_review;all_valid_seeds_at_selected_lr_retained;posthoc_user_lr_override;fixed_seed123_retained;absolute_mean_spearman_below_0.1;nonpositive_seed_spearman; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","posthoc_top3_test_selection;posthoc_user_lr_override;std_above_0.1;not_unbiased_fixed_seed_repetitions","posthoc_top3_test_selection;posthoc_user_lr_override;not_unbiased_fixed_seed_repetitions;small_test_set_n36;discrete_predictions","technical_failure_seed_replacement;accepted_after_test_review;all_valid_seeds_at_locked_lr_retained;all_positive_class_predictions;zero_std_from_class_collapse;finite_f1_not_discrimination_evidence","accepted_after_test_review;posthoc_test_auroc_tiebreak_selection;all_binary_f1_tied;all_predictions_class1;balanced_accuracy_0.5;mcc_0;zero_f1_std_is_degeneracy_not_stability;class_imbalance;discrete_near_constant_scores;fp16;filter_lr_fixed_6e-4","Fixed gene-disjoint split; class/distribution changes; limited held-out genes; LR transferred from old validation; no new-split LR search","Fixed gene-disjoint split; class/distribution changes; limited held-out genes; Evo2 searches three LRs on new validation; BF16/FP8 numerical-update exception retained","Fixed gene-disjoint split; class/distribution changes; limited held-out genes; cardiovascular Caduceus-PS seeds12 BF16, seed3 FP16; LR transferred from old validation; no new-split LR search","Inherited Evo2 numerically inactive final-layer adaptation; fixed seeds1/2/3","test_informed_user_lr_override;mixed_precision_seed1_fp16_seed23_bf16","classification_selected_lr_transferred_without_regression_lr_search","classification_selected_lr_transferred_without_regression_lr_search;mixed_precision_seed1_bf16_seeds23_fp16_user_accepted","classification_selected_lr_transferred_without_regression_lr_search;user_accepted_seed234_after_observing_seed1_test_degeneracy;seed1_excluded_undefined_correlations_all10_near_constant;all_attempts_retained;checkpoint_selected_by_validation_loss","Classification LR transferred; BF16/FP8 numerical exception retained","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out;selected_seed1_reused;seeds23_autotune4;algorithm_change_can_affect_low_precision_rounding","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out","BF16/FP8 last-layer numerical-update exception retained; author split; sequence-only; first 8192 nt; user-fixed LR, no LR search","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out;two_extra_LRs_after_nonfinite_trials;test_integrity_used_for_eligibility_not_score_ranking","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out;outer0_public_fold;inner0_reconstructed_public_order;scalar_mean_observed_tracks","human_author_split;sequence_only;validation_LR_selection;fixed_seeds123;10epochs;not_homology_held_out;two_extra_LRs_after_nonfinite_trials;test_integrity_used_for_eligibility_not_score_ranking;outer0_public_fold;inner0_reconstructed_public_order;scalar_mean_observed_tracks","old_release_validation_lr_transferred_by_user;","old_release_validation_lr_transferred_by_user;;micro_batch_differs_across_seeds; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","current_release_minimum_validation_loss;;seed1_fp16_seed23_bf16_user_retry;sd_not_pure_seed_variation","current_release_minimum_validation_loss;","current_release_minimum_validation_loss;;random_init_baseline_max_epochs30","old_release_validation_lr_transferred_by_user;; Post-hoc test-informed seed selection after six extra-seed batches, accepted on 2026-10-06. Four explicit user subsets are retained; otherwise highest three finite test scores among audited retained original and planned extra seeds. LR and dataset splits unchanged. This subset SD is not a prespecified unbiased three-seed stability estimate. All raw runs retained.","Inherited Evo2 numerically inactive final-layer adaptation; fixed seeds1/2/3; seeds2/3 resumed with native strict model/optimizer/precision/RNG restoration from durable checkpoints","current_release_minimum_validation_loss;Excluded seed1 LR1e-3: documented NaN failure; selected between two successful finite validation histories","old_release_validation_lr_transferred_by_user;;random_init_baseline_max_epochs30"]}