Linux configuration settings for Sybase ASE
To run Sybase ASE successfully on a Linux environment you need to configure a number of items. Below you will find a few of these, but please note that this is by far not an full overview. There could be more improvements to be made.
Contents
Memory – Huge Pages
The configuration of Huge Pages is needed to get an efficient translation from virtual memory pages to physical pages. When you configure for Huge Pages, the amount of memory assigned to Huge Pages is allocated and reserved for shared memory. Do not over configure to prevent swapping.
When ASE reports the below message in the errorlog, you need to configure for huge pages:
Could not allocate memory using Huge Pages. Allocated using regular pages. For better performance, reboot the server after configuring enough Huge Pages.
To get the values for huge pages first determine the maximum amount of memory you want to give to ASE. For instance, on a Linux server with 512Gb physical memory you can give 90% to ASE. 512 Gb x 90% = 460 Gb = 471040 Mb.
Then determine the size of a single huge page:
cat /proc/meminfo | grep Hugepagesize
Sample output:
Hugepagesize: 2048 kB
In this case 1 huge page is 2 Mb, so the number of huge pages we need will be 471040 Mb / 2 Mb = 235520
Then set the correct value in /etc/sysctl.conf:
vm.nr_hugepages = <value>
Next step is to calculate the number of huge pages in Kb and give the “sybase” user permission to grab the memory. Take the number of huge pages as calculated above. Then multiply by 2048. That wil give you the number of pages in Kb. Example: 235520 x 2048 = 482344960. Then have it stored in /etc/security/limits.conf like this
sybase soft memlock 482344960 sybase hard memlock 482344960
In this example, "sybase" is assumed to be the unix user running ASE.
Now reboot the Linux server to check if everything works as expected.
To see the configured number of huge pages at runtime:
cat /proc/sys/vm/nr_hugepages
Notes:
- As per Sybase recommendation (CR 627114) “lock shared memory” must be switched off.
- When ASE uses huge pages, it will allocate memory to the nearest multiple of 256Mb so don’t configure the ASE exactly to the maximum number of huge pages, but leave a small amount unused (e.g. 300 Mb). In case a small increase in size is needed due to the change of a Sybase config you will not get into trouble.
Number of file descriptors
You may need to increase the number of file descriptors per process if you get this error:
The number of file descriptors per ASE process required to support the current configuration exceeds the limit 4096 by 3955.
Make the following changes to /etc/security/limits.conf
sybase soft nofile 16384 sybase hard nofile 16384
In this example, "sybase" is assumed to be the unix user running ASE.
Log off and log on again to pick up the new settings.
DISK I/O
fs.aio-max-nr
fs.aio-max-nr is the maximum number of asynchronous I/Os.
The recommended minimum value for fs.aio-max-nr is 1048576, but on a busy ASE with many engines you may need to have a higher number configured.
In ASE, set these values as a starting point.
- Set “disk i/o structures” to 1048576.
- Set “max async i/os per engine” to 8192.
- Keep "max async i/os per server" at it’s default value (2147483647).
When you have multiple engines configured within ASE then use this formula to calculate the value for fs.aio-max-nr
(disk i/o structures + max async i/os per engine) * max online engines
For example: (1048576 + 8192) * 32 engines = 33816576
Then set the value in /etc/sysctl.conf:
fs.aio-max-nr = 33816576
To get the value at runtime
cat /proc/sys/fs/aio-max-nr
Troubleshooting fs.aio-max-nr
When the below messages appear in the ASE errorlog, the configuration of fs.aio-max-nr is not correct:
kernel asynchronous I/O not initialized. The io_setup() system call returned -22.
or
kernel kistartup: asynchronous I/O not available: No such file or directory
Either recalculate the vale or reduce the Sybase related configuration options (“disk i/o structures” and/or “max async i/os per engine”)
Troubleshooting dstartio
When this message appears in the errorlog
kernel dstartio: I/O request repeatedly delayed; block number: <nr>, vdn: <nr>
then you need to increase "disk i/o structures" or "max async i/os per engine".
To determine if "disk i/o structures" or "max async i/os per engine" needs to be increased look at the output of sp_sysmon that was captured during the time that the message appeared. Snapshot of output:
I/Os Delayed by Disk I/O Structures n/a n/a 0 n/a Server Config Limit n/a n/a 0 n/a Engine Config Limit n/a n/a 24582 n/a Operating System Limit n/a n/a 0 n/a
When a non-zero value is reported, increase the corresponding Sybase parameter. Alternatively, you can run sp_monitorconfig “all” to see if the maximum number of “disk i/o structures” has reached (see column Max_Used).
Be aware that the Linux setting for fs.aio-max-nr may need to be increased as well after changing the Sybase configuration settings.
Queue depth
Per Linux volume used by ASE the queue depth must be configured. The default value is 128 but it must be set to 1024.
You can only set the required value at boottime and, depending on the Linux distribution, you need to do it in /etc/init.d/boot.local (Suse) or /etc/init.d/rc.local (Ubuntu) or /etc/rc.d/rc.local (RedHat)
For example:
echo 1024 >/sys/block/sdb/queue/nr_requests
To retrieve the runtime value:
cat /sys/block/sd*/queue/nr_requests
kernel.randomize_va_space
This may be Suse and ASE 15.0.3 specific, but it may apply to other Linux / ASE version combinations as well.
Due to a high I/O load you may see stacktraces referring to the cm_writedes module, see below snippet.
00:00000:00006:2012/11/11 01:18:41.45 kernel pc: 0x00007ffff73a56a0 (null)+0x7ffff73a56a0() 00:00000:00006:2012/11/11 01:18:41.45 kernel pc: 0x00000000018fd036 cm_writedes+0x606()
In the later versions of ASE 15.7 this should be fixed and an error message is printed. To prevent the issue, set the value for kernel.randomize_va_space to 0 in combination with a higher value of “cpu grace time” in ASE, like 1500. To see the runtime value
cat /proc/sys/kernel/randomize_va_space
To see the boottime value, check /etc/sysctl.conf there must be line like kernel.randomize_va_space = 0
See also http://www.novell.com/support/kb/doc.php?id=7004855
After changing kernel.randomize_va_space do a reboot of the Linux server, as that seems to be necessary.
Prevent a firewall from dropping idle connections
For connections to ASE
When your Sybase server is behind a firewall there is always the risk that a connection from the client side (e.g. running isql) to ASE will be dropped when the firewall sees it as an idle session. This is regardless whether the session is really idle or that it is just a very long running query. To prevent it, put this in /etc/sysctl.conf
net.ipv4.tcp_keepalive_time = 300
and activate it
sysctl -p
For ssh sessions
Putty
Enable TCP keepalive. On the start screen of Putty click "Connection" and activate the check-box "Enable TCP keepalives".
ssh
Create a config file in your ssh directory (~/.ssh/config) with the following content
Host * ServerAliveInterval 60 ServerAliveCountMax 1000 TCPKeepAlive=yes
Whenever you use ssh tell it to use the config file
ssh -F ~/.ssh/config <server>
Soft / Hard limits for memory allocation (Suse only)
By default, no more than 80 % of available memory (RAM and swap) can be claimed by a single process. This is the soft-virtual limit. To check the setting: cat /etc/sysconfig/ulimit and look for the value specified as SOFTVIRTUALLIMIT. Default is “80”.
Increase the setting as setting as needed, or set to “0” for unlimited.
Associated Sybase error message:
Attempt to attach shared memory (id 32768) at address (nil), with flags 0 failed
