Assistance setting up cluster and encryption

Help
1

I’m in the process of setting up a cluster with 2 nodes.

I’m using a yaml file for the nodes that looks like this:

license: **************
high_availability: false
memsql_server_version: 7.3
package_type: deb
optimize: true
hosts:
- hostname: <agg_node_hostname>
  localhost: true
  nodes:
  - register: false
    role: Master
    config:
      password: **************
      port: 3306
- hostname: <leaf_node_hostname>
  localhost: false
  ssh:
    host: <leaf_node_hostname>
    private_key: **************
    user: **************
  nodes:
  - register: false
    role: Leaf
    config:
      password: **************
      port: 3306
      datadir: /mnt/data

I use the command: sdb-deploy setup-cluster --cluster-file /tmp/memsql.yaml

This appears to be successful. No errors and when I login I can see the cluster. When I connect using memsql -u root -p I can see the configuration and it appears to be fine.

Where I seem to be having issues is with trying to enable encryption using the following commands from the linux prompt:

sdb-admin update-config --all -y --key ssl_cert --value /path/to/cert
sdb-admin update-config --all -y --key ssl_cert --value /path/to/key
sdb-admin restart-node --all -y

When I try any of these commands I get the following:

Failed to check if 'ssl_cert' is a sync var.  Will attempt to update-config assuming that it is not.  Error is no master aggregator found in this cluster
Toolbox is about to run 'memsqlctl update-config --key ssl_cert --value /usr/local/bluetriangletech.pem' on the following nodes:
Would you like to continue? [Y/n]: 
Automatically selected yes, non-interactive mode enabled
Operation completed successfully

When I try to run sdb-admin list-nodes
I get:
No nodes found

I’ve had a really hard time trying to find how to do this.

Is there somewhere in the documentation that just runs through the basic setup to get sdb-admin aware of the cluster generated from the yaml file? Everything I’ve seen so far just ends up being for kubernetes or some other method of installing.

2

Hi Garret,

Typically that error message indicates that you’re running that command as a different linux user than the one you used to deploy.

If that’s not the issue, report back and we’ll try something else.

1 Like
3

That makes total sense in this case. We are using ansible to deploy memsql and ansible uses a user/key specific to the memsql install process. When we go in to do maintenance or just check the status of the instance we use our individual user/keys. Is there any way to get around this? It seems strange that we have to use the specific user in order to be able to use the sdb-admin.

I apologize for my ignorance on this. I am not that familiar with memsql/singlestore. I see that using memsqlctl instead of sdb-admin works!

4

Ok, glad to see you got through via memsqlctl.

I’ll ask around about your Ansible/etc. question.

  • Garrett
5

Ok, we may be getting to the edge of what we can help with via the forum, but:

The approach is for all users to have toolbox.hcl (path findable via sdb-admin env) files pointing to the same statefile/etc and the users would need to be in the same group with all toolbox and memsqlctl related files having group write/read/exec privs

Also should need to be in the memsql group.

6

Thanks again for the help. I’ve got the commands working now but unfortunately I can’t get the ssl certificates working.

Could you provide a link to instructions on how to configure it? I’ve been all over the support documentation and

I’m running the following:
memsqlctl update-config --all -y --key ssl_key --value /location/of/key-file
memsqlctl update-config --all -y --key ssl_cert --value /location/of/crt-file
memsqlctl restart-node --all -y

When it does the restart, it fails with the following:

Latest errors from MemSQL tracelog:
55 2021-08-05 19:12:41.966 INFO: Log opened
01753215 2021-08-05 19:12:43.719  ERROR: SSL load certificate file failed with Permission denied (13)
01753244 2021-08-05 19:12:43.719  ERROR: SSL load certificate file failed with SSL_CTX_use_certificate_file:system 
lib (0)
01753254 2021-08-05 19:12:43.719  ERROR: SSL setup failed using file: �� ��|�&�Jh@��sk���?z    %r_�V*�P�2�膩� &&�
A#K�Ѝ����K���,@l���趪�ҍ��]���C��^7��Az�{�aXž�l�f/��j���A���M�Ѐ|1�ֵ�!ܸT�.6�qe0���
01753267 2021-08-05 19:12:43.719  ERROR: Failed to start MemSQL
01753276 2021-08-05 19:12:43.719  ERROR: ./memsqld: exiting
42 2021-08-05 19:14:52.039 INFO: Log opened
01752173 2021-08-05 19:14:53.791  ERROR: SSL load key file failed with Connection timed out (110)
01752203 2021-08-05 19:14:53.791  ERROR: SSL load key file failed with SSL_CTX_use_PrivateKey_file:PEM lib (0)
01752214 2021-08-05 19:14:53.791  ERROR: SSL setup failed using file: /location/of/crt-file
01752226 2021-08-05 19:14:53.791  ERROR: Failed to start MemSQL
01752235 2021-08-05 19:14:53.791  ERROR: ./memsqld: exiting
45 2021-08-05 19:33:50.133 INFO: Log opened
01751631 2021-08-05 19:33:51.885  ERROR: SSL load key file failed with Connection timed out (110)
01751663 2021-08-05 19:33:51.885  ERROR: SSL load key file failed with SSL_CTX_use_PrivateKey_file:PEM lib (0)
01751684 2021-08-05 19:33:51.885  ERROR: SSL setup failed using file: /location/of/crt-file
01751697 2021-08-05 19:33:51.885  ERROR: Failed to start MemSQL
01751706 2021-08-05 19:33:51.885  ERROR: ./memsqld: exiting
: Failed to connect to MemSQL: process exited: exit status 1

I’ve gone through the steps in the video here:

Any thoughts?

7

Just want to double check and verify that you’re entering the correct path to the cert and key. I’m pretty sure it’s a relative path. (I’m assuming you’re ‘sanitizing’ your output above)

Also, permission denied? Does the user you’re running as have access permission to the cert file?

Verify those and then we can look at other options.

-Garrett

8

There was an issue with the key file.

All is good now.
Thanks! You guys rock.
Garret