Exit after certain number of requests.

[ribalba]

Hello,

at Green Coding Berlin [1] we create software/ tooling to measure the energy usage of software. I had an idea to compare various DBMS against each other not in TPM but in energy usage while doing the same workload. So I want to run a few databases with the same workload and see how they compare in resource usage. Now I started to test out HammerDB and I can get the standard benchmark to run in our setup. Thanks to the great docker container. But I can't find a way to benchmark each database with the same amount of "work". If I use the timed driver it will try and do as many queries as possible in the time frame. Which wouldn't be a good comparison as if DB1 is faster in processing queries to DB2 it will have done more and such the energy measurements are not comparable. I could read out the TPM and then somehow reference them back to energy usage but that will be quite an overhead for a little demo.

Does someone have an idea how I can tell HammerDB to only execute n requests and then stop or just do nothing?
I have tried playing around with pg_total_iterations and vuset iterations 10 but none really do what I need. Thank you for any pointers or help.

[1] https://www.green-coding.berlin/

[sm-shaw]

Hi,
Yes, you want to use a fixed throughput approach that is described here https://www.hammerdb.com/blog/uncategorized/how-to-run-a-fixed-throughput-workloads/
Then you can scale up the throughput according to the number of connections.
It will be the NOPM (schema based metric) that is fixed rather than the TPM (database based metric) - some of the implementations vary slightly in terms of the number of database commits that provide the most efficient throughput.

[ribalba]

Thank you for your feedback. I already tried to set key pg_keyandthink to true. But it seems like

Value true for tpcc:pg_keyandthink is the same as existing value true, no change made

And I still get 19836 PostgreSQL tpm.

Maybe I am missing something? Here is my config file:

#!/bin/tclsh
# maintainer: Pooja Jain

puts "SETTING CONFIGURATION"
dbset db pg
dbset bm TPC-C

diset connection pg_host postgres_container
diset connection pg_port 5432
diset connection pg_sslmode disable

diset tpcc pg_superuser postgres
diset tpcc pg_superuserpass postgres
diset tpcc pg_defaultdbase postgres
diset tpcc pg_user postgres
diset tpcc pg_pass postgres
diset tpcc pg_dbase postgres

diset tpcc pg_driver timed

diset tpcc pg_keyandthink true


diset tpcc pg_rampup 2
diset tpcc pg_duration 2

diset tpcc pg_vacuum true
diset tpcc pg_timeprofile true
diset tpcc pg_allwarehouse true

loadscript
puts "TEST STARTED"
vuset vu vcpu
#vuset iterations 1
vuset vu 1
print vuconf

vucreate
tcstart
tcstatus
set jobid [ vurun ]
vudestroy
tcstop
puts "TEST COMPLETE"
set of [ open /tmp/pg_tprocc w ]
puts $of $jobid
close $of

and the output I am getting

$ docker exec hammerdb_container ./hammerdbcli auto /tmp/r.tcl 
HammerDB CLI v4.8
Copyright (C) 2003-2023 Steve Shaw
Type "help" for a list of commands
Initialized Jobs on-disk database /tmp/hammer.DB using existing tables (208,896 KB)
SETTING CONFIGURATION
Database set to PostgreSQL
Benchmark set to TPC-C for PostgreSQL
Value postgres_container for connection:pg_host is the same as existing value postgres_container, no change made
Value 5432 for connection:pg_port is the same as existing value 5432, no change made
Value disable for connection:pg_sslmode is the same as existing value disable, no change made
Value postgres for tpcc:pg_superuser is the same as existing value postgres, no change made
Value postgres for tpcc:pg_superuserpass is the same as existing value postgres, no change made
Value postgres for tpcc:pg_defaultdbase is the same as existing value postgres, no change made
Value postgres for tpcc:pg_user is the same as existing value postgres, no change made
Value postgres for tpcc:pg_pass is the same as existing value postgres, no change made
Value postgres for tpcc:pg_dbase is the same as existing value postgres, no change made
Value timed for tpcc:pg_driver is the same as existing value timed, no change made
Value true for tpcc:pg_keyandthink is the same as existing value true, no change made
Value 2 for tpcc:pg_rampup is the same as existing value 2, no change made
Value 2 for tpcc:pg_duration is the same as existing value 2, no change made
Value true for tpcc:pg_vacuum is the same as existing value true, no change made
Value true for tpcc:pg_timeprofile is the same as existing value true, no change made
Value true for tpcc:pg_allwarehouse is the same as existing value true, no change made
Script loaded, Type "print script" to view
TEST STARTED
Virtual Users = 1
User Delay(ms) = 500
Repeat Delay(ms) = 500
Iterations = 1
Show Output = 1
Log Output = 0
Unique Log Name = 0
No Log Buffer = 0
Log Timestamps = 0
Vuser 1 created MONITOR - WAIT IDLE
Vuser 2 created - WAIT IDLE
2 Virtual Users Created with Monitor VU
Transaction Counter Started
Transaction Counter thread running with threadid:tid0x7fa82a648700
Vuser 1:RUNNING
Vuser 1:Initializing xtprof time profiler
Vuser 1:Beginning rampup time of 2 minutes
0 PostgreSQL tpm
Vuser 2:RUNNING
Vuser 2:Initializing xtprof time profiler
Vuser 2:VU 2 : Assigning WID=1 based on VU count 1, Warehouses = 1 (1 out of 1)
Vuser 2:Processing 100000 transactions with output suppressed...
19836 PostgreSQL tpm
19782 PostgreSQL tpm
^C

[sm-shaw]

It looks like something else is still running. The TPM will show all transactions against your database, so it could be another workload you ran earlier is still running.
in this case you have set #vuset iterations 1 and even though it is commented out it will persist this value until you set it again. This means your virtual user will run 1 transaction and then finish the workload, regardless of the ramp-up and duration time. You need to set the iterations back to a value such as 1000000. You then need to set asynch_scale and async_cllient to create a larger number of connections. The blog post will guide you on what to do to get an effective fixed throughput workload.

[ribalba]

Thank you for your help and feedback. I couldn't get this working in the end. We are now going to take the NOPM as unit of work and split up the total power consumption by this. Like this we can use the std run files. Cheers