When can I remove old txn logs?

Help
1

Hello,

When is it safe to remove old txn logs? I’ve run snapshots periodically, but how soon after a snapshot will the logs files be useful to keep around?
.
.
.
-rw------- 1 memsql memsql 268435456 Apr 20 12:45 rh_test_3_log_v1_327680
-rw------- 1 memsql memsql 268435456 Apr 16 19:11 rh_test_3_log_v1_393216
-rw------- 1 memsql memsql 268435456 Apr 16 17:51 rh_test_3_log_v1_458752
-rw------- 1 memsql memsql 268435456 Apr 16 18:31 rh_test_3_log_v1_524288
-rw------- 1 memsql memsql 268435456 Apr 16 19:11 rh_test_3_log_v1_589824
-rw------- 1 memsql memsql 268435456 Apr 20 12:45 rh_test_4_log_v1_327680
-rw------- 1 memsql memsql 268435456 Apr 16 19:16 rh_test_4_log_v1_393216
-rw------- 1 memsql memsql 268435456 Apr 16 17:53 rh_test_4_log_v1_458752
-rw------- 1 memsql memsql 268435456 Apr 16 18:35 rh_test_4_log_v1_524288
-rw------- 1 memsql memsql 268435456 Apr 16 19:16 rh_test_4_log_v1_589824
.
.
.

2

Hello,

You should never remove old transaction logs - memsql does that automatically. If you delete them manually, you’ll likely end up losing or corrupting your data. The amount of log file content that is kept around is dependent on snapshots_to_keep - memsql deletes all log content below the lowest snapshot. For performance reasons, however, the log files are pre-sized (in this case you’re using the default 256MB size for partitions, which is settable at create database time), and extra unused files are kept around - a minimum of 1 unused file, and maximum of 4 unused files, the actual number dynamically depends on your workload.

If you need your log files to take up less space, then you can adjust the globals log_file_size_partitions and log_file_size_ref_dbs (these default to 256MB and 64MB respectively) before creating the database, or restoring it from a backup - at this time it is impossible to change the log file size for a database without restoring it from a backup.

3

Thank you Rodrigo for the explanation, it was very helpful.

1 Like
4

Follow up to the txn logs: Why would there be a need to keep a large number of logs per partition?
-rw------- 1 memsql memsql 268435456 Apr 22 16:06 hl7_7_log_v1_2490368
-rw------- 1 memsql memsql 268435456 Apr 22 16:21 hl7_7_log_v1_2555904
-rw------- 1 memsql memsql 268435456 Apr 22 16:35 hl7_7_log_v1_2621440
-rw------- 1 memsql memsql 268435456 Apr 22 17:03 hl7_7_log_v1_2686976
-rw------- 1 memsql memsql 268435456 Apr 22 17:14 hl7_7_log_v1_2752512
-rw------- 1 memsql memsql 268435456 Apr 22 17:48 hl7_7_log_v1_2818048
-rw------- 1 memsql memsql 268435456 Apr 22 17:58 hl7_7_log_v1_2883584
-rw------- 1 memsql memsql 268435456 Apr 22 18:09 hl7_7_log_v1_2949120
-rw------- 1 memsql memsql 268435456 Apr 22 18:18 hl7_7_log_v1_3014656
-rw------- 1 memsql memsql 268435456 Apr 23 08:13 hl7_7_log_v1_3080192
-rw------- 1 memsql memsql 268435456 Apr 23 08:54 hl7_7_log_v1_3145728
-rw------- 1 memsql memsql 268435456 Apr 23 08:55 hl7_7_log_v1_3211264
-rw------- 1 memsql memsql 268435456 Apr 23 08:54 hl7_7_log_v1_3276800

The database show a size on disk of 8G, but the txn logs are consuming close to 26G.

5

Also, I see 71 16M log files for the memsql database on the leaf nodes:
-rw------- 1 memsql memsql 16777216 Apr 23 12:13 memsql_log_v1_282624
-rw------- 1 memsql memsql 16777216 Apr 23 10:50 memsql_log_v1_286720
.
.
.
Is that normal behavior?

6

Hi,

This is expected, depending on your snapshots_to_keep and snapshot_trigger_size. We keep extra logs on “standby”, but we also keep around logs that have valid content. Log content between snapshots is split into multiple fixed-size logs, so it’s not unusual to have many small log files.

As an example:
Suppose you have snapshot_trigger_size set to 2GB, snapshots_to_keep set to 2, and log_file_size_partitions set to 16MB. In that case, you would keep 2GB worth of log content between each snapshot you keep on disk, or up to 4GB per partition. That means each partition would keep up to 250 log files.

If you manually take snapshots, you can manually control how much space exists between each snapshot, forcing a cleanup of log files, but if you then let the system run, it’ll eventually create a bunch of new log files with log content.

If you want to reduce the total amount of space occupied by all log files, not just unused ones, you also need to adjust snapshots_to_keep and snapshot_trigger_size.

7

Can I change snapshot_trigger_size on a live cluster and have it take affect immediately?

8

Yes, snapshot_trigger_size changes are online, and take effect right away.

9

That helped immensely.

Thank you.

1 Like
10

Hi,

I have the following settings:
log_file_size_partitions = 10485760 (10MB)
snapshot_trigger_size = 536870912 (500MB)
snapshots_to_keep = 1

Yet there are a little over 300 10MB files in the data directory for one of the DB taking therefore about 3GB worth of disk space.
Could you explain why? I was expecting no more than 500MB worth of space taken by these files.

Thanks