Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add tproxy initializer for Integration Tests #1254

Merged
merged 1 commit into from
Nov 19, 2024
Merged

Add tproxy initializer for Integration Tests #1254

merged 1 commit into from
Nov 19, 2024

Conversation

jbesraa
Copy link
Contributor

@jbesraa jbesraa commented Nov 14, 2024

resolves #1250

@codecov Codecov
Copy link

codecov bot commented Nov 14, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.30%. Comparing base (67a5926) to head (1fea709).
Report is 2 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1254   +/-   ##
=======================================
  Coverage   19.30%   19.30%           
=======================================
  Files         164      164           
  Lines       10849    10849           
=======================================
  Hits         2094     2094           
  Misses       8755     8755
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.46% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (ø)
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.51% <ø> (ø)
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 1.38% <ø> (ø)
protocols 24.72% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.06% <ø> (ø)
sv1-mining-device-coverage 0.00% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (ø)
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 14, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-14-tproxy-test-initializer
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8,379.00
(-0.94%)		8,801.86
(95.20%)		📈 view plot
🚷 view threshold		3,689.00
(-1.57%)		3,881.85
(95.03%)		📈 view plot
🚷 view threshold		5,159.00
(-1.84%)		5,450.80
(94.65%)		📈 view plot
🚷 view threshold		7.00
(-2.71%)		13.01
(53.82%)		📈 view plot
🚷 view threshold		91.00
(+0.56%)		95.87
(94.92%)
get_submit📈 view plot
🚷 view threshold		95,407.00
(-0.02%)		95,958.18
(99.43%)		📈 view plot
🚷 view threshold		59,346.00
(-0.19%)		59,699.54
(99.41%)		📈 view plot
🚷 view threshold		85,222.00
(-0.21%)		85,770.34
(99.36%)		📈 view plot
🚷 view threshold		42.00
(-6.59%)		63.98
(65.64%)		📈 view plot
🚷 view threshold		285.00
(+1.80%)		289.36
(98.49%)
get_subscribe📈 view plot
🚷 view threshold		7,872.00
(-1.79%)		8,357.07
(94.20%)		📈 view plot
🚷 view threshold		2,765.00
(-2.43%)		2,962.24
(93.34%)		📈 view plot
🚷 view threshold		3,847.00
(-2.88%)		4,144.74
(92.82%)		📈 view plot
🚷 view threshold		14.00
(+16.74%)		20.75
(67.45%)		📈 view plot
🚷 view threshold		113.00
(-1.00%)		119.67
(94.42%)
serialize_authorize📈 view plot
🚷 view threshold		12,210.00
(-0.54%)		12,627.44
(96.69%)		📈 view plot
🚷 view threshold		5,266.00
(-1.00%)		5,451.07
(96.60%)		📈 view plot
🚷 view threshold		7,325.00
(-1.26%)		7,611.40
(96.24%)		📈 view plot
🚷 view threshold		11.00
(+20.55%)		16.13
(68.21%)		📈 view plot
🚷 view threshold		138.00
(+0.35%)		143.35
(96.27%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24,340.00
(-1.39%)		25,270.82
(96.32%)		📈 view plot
🚷 view threshold		9,838.00
(-0.53%)		10,048.99
(97.90%)		📈 view plot
🚷 view threshold		13,890.00
(-0.53%)		14,202.90
(97.80%)		📈 view plot
🚷 view threshold		32.00
(-7.83%)		44.11
(72.54%)		📈 view plot
🚷 view threshold		294.00
(-2.43%)		315.62
(93.15%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30,068.00
(-0.83%)		30,780.36
(97.69%)		📈 view plot
🚷 view threshold		12,015.00
(-0.55%)		12,219.64
(98.33%)		📈 view plot
🚷 view threshold		17,003.00
(-0.61%)		17,312.03
(98.21%)		📈 view plot
🚷 view threshold		58.00
(+4.18%)		67.98
(85.32%)		📈 view plot
🚷 view threshold		365.00
(-1.22%)		381.76
(95.61%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126,369.00
(-0.07%)		127,028.60
(99.48%)		📈 view plot
🚷 view threshold		73,200.00
(-0.11%)		73,570.56
(99.50%)		📈 view plot
🚷 view threshold		104,914.00
(-0.14%)		105,510.30
(99.43%)		📈 view plot
🚷 view threshold		105.00
(-1.93%)		133.33
(78.75%)		📈 view plot
🚷 view threshold		598.00
(+0.29%)		605.42
(98.77%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27,468.00
(-1.30%)		28,638.51
(95.91%)		📈 view plot
🚷 view threshold		9,584.00
(-0.66%)		9,779.75
(98.00%)		📈 view plot
🚷 view threshold		13,543.00
(-0.76%)		13,838.47
(97.86%)		📈 view plot
🚷 view threshold		69.00
(+8.69%)		76.06
(90.72%)		📈 view plot
🚷 view threshold		388.00
(-2.06%)		417.14
(93.01%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115,117.00
(-0.10%)		115,890.07
(99.33%)		📈 view plot
🚷 view threshold		68,060.00
(-0.06%)		68,421.70
(99.47%)		📈 view plot
🚷 view threshold		97,657.00
(-0.08%)		98,248.58
(99.40%)		📈 view plot
🚷 view threshold		62.00
(-1.15%)		78.25
(79.24%)		📈 view plot
🚷 view threshold		490.00
(-0.22%)		499.30
(98.14%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		22,842.00
(-1.74%)		24,053.70
(94.96%)		📈 view plot
🚷 view threshold		8,143.00
(-0.73%)		8,335.98
(97.68%)		📈 view plot
🚷 view threshold		11,467.00
(-0.76%)		11,744.97
(97.63%)		📈 view plot
🚷 view threshold		35.00
(-6.04%)		44.45
(78.74%)		📈 view plot
🚷 view threshold		320.00
(-2.66%)		348.72
(91.76%)
serialize_submit📈 view plot
🚷 view threshold		99,796.00
(-0.05%)		100,357.68
(99.44%)		📈 view plot
🚷 view threshold		61,408.00
(-0.16%)		61,736.95
(99.47%)		📈 view plot
🚷 view threshold		88,081.00
(-0.19%)		88,608.21
(99.41%)		📈 view plot
🚷 view threshold		47.00
(+0.35%)		64.69
(72.66%)		📈 view plot
🚷 view threshold		328.00
(+1.05%)		333.97
(98.21%)
serialize_subscribe📈 view plot
🚷 view threshold		11,317.00
(-0.71%)		11,742.18
(96.38%)		📈 view plot
🚷 view threshold		4,118.00
(-1.51%)		4,307.28
(95.61%)		📈 view plot
🚷 view threshold		5,707.00
(-1.91%)		5,999.49
(95.12%)		📈 view plot
🚷 view threshold		16.00
(+24.42%)		20.71
(77.27%)		📈 view plot
🚷 view threshold		158.00
(+0.26%)		163.88
(96.41%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 14, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-14-tproxy-test-initializer
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,649.70
(-0.76%)		7,073.99
(94.00%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,646.70
(+0.04%)		7,924.64
(96.49%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,208.20
(+0.13%)		8,553.42
(95.96%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		875.97
(+0.10%)		946.06
(92.59%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		683.11
(+1.39%)		723.71
(94.39%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		247.41
(-0.49%)		261.55
(94.59%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		157.67
(+0.04%)		163.65
(96.35%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,409.70
(-1.12%)		7,171.26
(89.38%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		273.71
(-1.22%)		297.56
(91.98%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		733.02
(-0.43%)		817.57
(89.66%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		626.10
(+4.02%)		669.19
(93.56%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		207.25
(+0.05%)		224.32
(92.39%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 14, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-14-tproxy-test-initializer
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.23
(-0.84%)		47.22
(93.67%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		74.63
(-1.02%)		84.29
(88.54%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.65
(-0.07%)		14.72
(99.56%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		269.56
(+1.32%)		316.40
(85.19%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		615.03
(+1.82%)		750.34
(81.97%)
client_sv2_open_channel📈 view plot
🚷 view threshold		147.10
(-0.36%)		156.94
(93.73%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		280.53
(-1.55%)		316.13
(88.74%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		389.11
(+2.91%)		415.15
(93.73%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		158.74
(-1.33%)		175.75
(90.32%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		490.17
(+4.66%)		573.42
(85.48%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		991.81
(+1.84%)		1,069.19
(92.76%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 14, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-14-tproxy-test-initializer
Testbedsv2

🚨 1 Alert

BenchmarkMeasure
Units		ViewBenchmark Result
(Result Δ%)		Upper Boundary
(Limit %)
client_sv2_mining_message_submit_standard_serialize_deserializeEstimated Cycles
estimated cycles		📈 plot
🚨 alert (🔔)
🚷 threshold		27,882.00
(+1.27%)		27,787.35
(100.34%)
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2,103.00
(+0.45%)		2,206.38
(95.31%)		📈 view plot
🚷 view threshold		473.00
(-0.07%)		488.52
(96.82%)		📈 view plot
🚷 view threshold		733.00
(-0.39%)		757.28
(96.79%)		📈 view plot
🚷 view threshold		8.00
(+46.92%)		14.11
(56.69%)		📈 view plot
🚷 view threshold		38.00
(-0.04%)		41.17
(92.30%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8,204.00
(-0.00%)		8,286.46
(99.00%)		📈 view plot
🚷 view threshold		2,137.00
(-0.03%)		2,143.08
(99.72%)		📈 view plot
🚷 view threshold		3,159.00
(-0.06%)		3,170.60
(99.63%)		📈 view plot
🚷 view threshold		36.00
(+1.25%)		42.18
(85.36%)		📈 view plot
🚷 view threshold		139.00
(-0.02%)		140.89
(98.66%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6,323.00
(+0.69%)		6,391.88
(98.92%)		📈 view plot
🚷 view threshold		1,750.00
(-0.05%)		1,766.88
(99.04%)		📈 view plot
🚷 view threshold		2,548.00
(-0.23%)		2,575.62
(98.93%)		📈 view plot
🚷 view threshold		20.00
(+17.92%)		24.54
(81.49%)		📈 view plot
🚷 view threshold		105.00
(+0.94%)		107.12
(98.02%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14,838.00
(+0.73%)		14,962.93
(99.17%)		📈 view plot
🚷 view threshold		4,694.00
(-0.02%)		4,710.88
(99.64%)		📈 view plot
🚷 view threshold		6,748.00
(-0.16%)		6,782.04
(99.50%)		📈 view plot
🚷 view threshold		50.00
(+14.86%)		53.51
(93.44%)		📈 view plot
🚷 view threshold		224.00
(+1.11%)		227.63
(98.41%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚨 view alert (🔔)
🚷 view threshold		27,882.00
(+1.27%)		27,787.35
(100.34%)		📈 view plot
🚷 view threshold		10,645.00
(+0.60%)		10,647.81
(99.97%)		📈 view plot
🚷 view threshold		15,502.00
(+0.68%)		15,502.80
(99.99%)		📈 view plot
🚷 view threshold		89.00
(+9.32%)		90.16
(98.71%)		📈 view plot
🚷 view threshold		341.00
(+1.77%)		341.64
(99.81%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4,409.00
(+0.30%)		4,605.96
(95.72%)		📈 view plot
🚷 view threshold		1,461.00
(-0.02%)		1,476.52
(98.95%)		📈 view plot
🚷 view threshold		2,159.00
(-0.04%)		2,184.41
(98.84%)		📈 view plot
🚷 view threshold		9.00
(+1.05%)		17.43
(51.65%)		📈 view plot
🚷 view threshold		63.00
(+0.61%)		68.23
(92.34%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14,046.00
(-0.08%)		14,431.12
(97.33%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,079.52
(99.69%)		📈 view plot
🚷 view threshold		7,326.00
(+0.00%)		7,352.03
(99.65%)		📈 view plot
🚷 view threshold		35.00
(-0.22%)		43.23
(80.95%)		📈 view plot
🚷 view threshold		187.00
(-0.16%)		198.06
(94.42%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22,854.00
(+0.84%)		22,903.41
(99.78%)		📈 view plot
🚷 view threshold		8,040.00
(+0.24%)		8,073.98
(99.58%)		📈 view plot
🚷 view threshold		11,689.00
(+0.18%)		11,753.01
(99.46%)		📈 view plot
🚷 view threshold		84.00
(+12.87%)		85.46
(98.30%)		📈 view plot
🚷 view threshold		307.00
(+1.15%)		311.41
(98.58%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4,681.00
(-0.07%)		4,772.88
(98.07%)		📈 view plot
🚷 view threshold		1,502.00
(-0.02%)		1,517.52
(98.98%)		📈 view plot
🚷 view threshold		2,276.00
(-0.11%)		2,299.13
(98.99%)		📈 view plot
🚷 view threshold		12.00
(+27.57%)		16.04
(74.83%)		📈 view plot
🚷 view threshold		67.00
(-0.58%)		69.80
(95.99%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16,190.00
(+0.12%)		16,458.18
(98.37%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,978.52
(99.74%)		📈 view plot
🚷 view threshold		8,660.00
(-0.04%)		8,691.71
(99.64%)		📈 view plot
🚷 view threshold		43.00
(+7.69%)		53.29
(80.69%)		📈 view plot
🚷 view threshold		209.00
(+0.10%)		216.26
(96.64%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35,722.00
(+0.48%)		35,753.45
(99.91%)		📈 view plot
🚷 view threshold		14,888.00
(+0.26%)		14,907.75
(99.87%)		📈 view plot
🚷 view threshold		21,882.00
(+0.31%)		21,920.23
(99.83%)		📈 view plot
🚷 view threshold		94.00
(+1.52%)		113.21
(83.03%)		📈 view plot
🚷 view threshold		382.00
(+0.72%)		384.65
(99.31%)
🐰 View full continuous benchmarking report in Bencher

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs
@@ -282,3 +286,86 @@ pub async fn start_template_provider(tp_port: u16) -> TemplateProvider {
template_provider.generate_blocks(16);
template_provider
}

pub async fn start_sv2_translator(upstream: SocketAddr) -> SocketAddr {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try to keep starter APIs consistent, as highlighted in #1234

I'm not opinionated towards any specific approach

I just feel it's important to be careful in fragmenting patterns because that will eventually result in a poor UX for SRI contributors writing tests while working on this like e.g.: #1229 (which will likely happen sooner, rather than later)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the approach here is mostly what we want. No need for the end user to call get_available_address but the role itself does that and returns it

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs
listening_address
}

fn measure_hashrate(duration_secs: u64) -> f64 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting approach of dynamically measuring local CPU hashrate

this strategy will save us A LOT of headache and it's quite a game changer in comparison to the static nature of MG tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Credit to @GitGab19

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs
.expect("failed");
let listening_address = get_available_address();
let listening_port = listening_address.port();
let hashrate = measure_hashrate(3) as f32 / 100.0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why divide by 100?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done just part of the testing, I think this will change also until we figure out the best numbers here and in measure_hashrate

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs Outdated
let mut hashes: u64 = 0;
let duration = std::time::Duration::from_secs(duration_secs);

let hash = |share: &mut [u8; 80]| -> Target {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why return Target, it seems to be used nowhere?

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs Outdated
let mut nonce = u64::from_le_bytes(nonce);
nonce += 1;
share[0..8].copy_from_slice(&nonce.to_le_bytes());
let hash = Sha256::digest(&share).to_vec();
Copy link
Collaborator

@plebhash plebhash Nov 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bitcoin protocol uses sha256d, which means we actually calculate sha256 2x

every ASIC hashrate will tell you how many sha256d hashes were calculated in a second, not sha256

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's an easy way to calculate sha256d hashes with stratum_common:

use stratum_common::bitcoin::hashes::{sha256d, Hash, HashEngine};

let input = 0;
let mut engine = sha256d::Hash::engine();
engine.input(&input);
let hashed = sha256d::Hash::from_engine(engine).into_inner();

plebhash
plebhash reviewed Nov 15, 2024
View reviewed changes
roles/tests-integration/tests/common/mod.rs
Comment on lines +340 to +445
let mut share = {
let mut rng = thread_rng();
let mut arr = [0u8; 80];
rng.fill(&mut arr[..]);
arr
};
let start_time = std::time::Instant::now();
let mut hashes: u64 = 0;
let duration = std::time::Duration::from_secs(duration_secs);
Copy link
Collaborator

@plebhash plebhash Nov 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this approach but it seems a bit convoluted and confusing to read/understand

I'm not convinced we really need to replicate the bitcoin protocol so closely if all we want is to know the capacity of the system to calculate sha256d (attention to the d, it's 2x sha256)

why not simply hash some simple input?

if you REALLY want to closely replicate what happens on bitcoin mining, I would suggest leveraging rust-bitcoin

you can initialize a Header struct and then call header.block_hash() to "mine" it.

that's how both mining-device and sv1-mining-device do it

@jbesraa
Copy link
Contributor Author

jbesraa commented Nov 17, 2024

@plebhash the measure_hashrate function and any function it uses is just a copy-pasta from a code @GitGab19 previously wrote. This pull request only uses this functionality without making changes to it yet. I think we should leave those change to be examined in the actual testing pull requests that will follow up.

@jbesraa jbesraa marked this pull request as ready for review November 18, 2024 09:56
@plebhash
Copy link
Collaborator

plebhash commented Nov 19, 2024 •
edited
Loading

@plebhash the measure_hashrate function and any function it uses is just a copy-pasta from a code @GitGab19 previously wrote. This pull request only uses this functionality without making changes to it yet. I think we should leave those change to be examined in the actual testing pull requests that will follow up.

I must point out that my comments about sha256d above are not purely aesthetic.

in it's current form, this measure_hashrate function is going to provide a wrong hashrate estimation because it's only calculating sha256, not sha256d (which is what actually happens on Bitcoin mining, and therefore the fundamental definition of hashrate).

I don't think we should introduce something with broken functionality to fix later, especially given that it's pretty straightforward to fix it now.

please see #1254 (comment)

@plebhash
Copy link
Collaborator

plebhash commented Nov 19, 2024 •
edited
Loading

after discussing with @GitGab19 (he did not write this code) we tracked down the root cause of the confusion around measure_hashrate function:

stratum/roles/translator/src/lib/downstream_sv1/diff_management.rs

Lines 374 to 400 in 8c98647

fn measure_hashrate(duration_secs: u64) -> f64 {
let mut share = generate_random_80_byte_array();
let start_time = Instant::now();
let mut hashes: u64 = 0;
let duration = Duration::from_secs(duration_secs);
while start_time.elapsed() < duration {
for _ in 0..10000 {
hash(&mut share);
hashes += 1;
}
}
let elapsed_secs = start_time.elapsed().as_secs_f64();
hashes as f64 / elapsed_secs
}
fn hash(share: &mut [u8; 80]) -> Target {
let nonce: [u8; 8] = share[0..8].try_into().unwrap();
let mut nonce = u64::from_le_bytes(nonce);
nonce += 1;
share[0..8].copy_from_slice(&nonce.to_le_bytes());
let hash = Sha256::digest(&share).to_vec();
let hash: U256<'static> = hash.try_into().unwrap();
hash.into()
}

we should not blindly copy-paste this code because it will result in a broken ITF

I'm taking note of this on #1229 so it can be fixed as we modularize vardiff (which is what these unit tests are for)

@plebhash plebhash mentioned this pull request Nov 19, 2024
Open
@plebhash plebhash changed the title Add tproxy initializer Add tproxy initializer for Integration Tests Nov 19, 2024
@jbesraa jbesraa force-pushed the 2024-11-14-tproxy-test-initializer branch from 3425383 to ada6c75 Compare November 19, 2024 15:00
@jbesraa jbesraa force-pushed the 2024-11-14-tproxy-test-initializer branch from ada6c75 to 1fea709 Compare November 19, 2024 15:34
@plebhash plebhash merged commit 0520c58 into stratum-mining:main Nov 19, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@plebhash plebhash plebhash approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add tproxy initializer in integration tests
2 participants
@jbesraa @plebhash