MENU

Drop Down MenusCSS Drop Down MenuPure CSS Dropdown Menu

Sunday, December 13, 2015

Trace file approach in Hana



The following sections of those trace files are usually relevant:
[IPMM_MEMORY]

The first part of this section lists the local (heap) memory the processes that make up the HANA database are currently using:

[0] PID=34884, SId=42309944, compactors active, alive, process name: hdbnameserver
AB=2220466176b (2.06gb), UA=0b, U=2015851859b (1.87gb), FSL=0b, PAL=487738426982b (454.24gb), TPAL=2220466176b (2.06gb), W=100

[1] PID=35049, SId=42310545, compactors active, alive, process name: hdbpreprocessor
AB=365772800b (348.82mb), UA=0b, U=362430594b (345.64mb), FSL=0b, PAL=487738426982b (454.24gb), TPAL=365772800b (348.82mb), W=100

[3] PID=35094, SId=42310846, compactors active, alive, process name: hdbstatisticsse
AB=17623138051b (16.41gb), UA=0b, U=14624613181b (13.62gb), FSL=268435456b (256mb), PAL=27096579276b (25.23gb), TPAL=17757355779b (16.53gb), W=100

[4] PID=35114, SId=42310947, compactors active, alive, process name: hdbxsengine
AB=2270855168b (2.11gb), UA=0b, U=2136436039b (1.98gb), FSL=0b, PAL=487738426982b (454.24gb), TPAL=2270855168b (2.11gb), W=100

[5] PID=33976, SId=171197412, compactors active, alive, process name: hdbindexserver
AB=240495694077b (223.97gb), UA=0b, U=260528715346b (242.63gb), FSL=0b, PAL=487738426982b (454.24gb), TPAL=270062153728b (251.51gb), W=100


For each process, the allocation limit (PAL), the amount of memory currently allocated (AB) and used (U) are displayed. An AB value that is significantly larger than an U value does not necessarily mean a problem: If necessary, allocated, but not used memory will be released. Of particular interest are lines where AB, U and PAL have approximately the same value. This particular process has then reached its allocation limit. Further analysis based on the output of M_HEAP_MEMORY in section [PROCESS_INFO] has to be done then.

The second part of section [IPMM_MEMORY] contains information regarding the shared memory:

GLOBAL_MAX_ALLOCATION_LIMIT=487738426983b (454.24gb), cached sum of allocation limits=487604209255b (454.11gb)
#checks for cached allocation limit sum=838859, #terminated processes since last reset=7
#processes=5, sum of temp process allocation limits=292676603651b (272.57gb), cached sum=487604209255b (454.11gb), weight sum=500, cached weight sum=500
SHARED_MEMORY=194927605604b (181.54gb), temp shared memory allocation limit=194927605604b (181.54gb)
IPMM reservation=0b, emergency reservation=134217728b (128mb)
112 early exits since creation of IPMM, provide-memory-counter=50279
Provide memory 50267 failed, only 109174531b were free.
Users respected in shm calculation: [302]

Shared Memory is used for various purposes. For practical purposes, it is sufficient to assume that mainly row store tables occupy shared memory. In case of the 'early exits since' output in the oom trace, the error was triggered by the inability to further increase the TPAL (temporary allocation limit) of the failing process. The allocation limit of a process is not set to the maximum right at process start, but increased (and decreased) over time. In this specific case, 'sum of temp process allocation limits' + SHARED_MEMORY exceeds the GLOBAL_MAX_ALLOCATION_LIMIT which consequently causes the error.

[MEMORY_OOM]

Before any detailed analysis is done, the information from section [MEMORY_OOM] has to be reviewed as well. It might be the case that the request was that large that it was anyway impossible to fulfill it:

[MEMORY_OOM]     Information about current out of memory situation:
OUT OF MEMORY occurred.
Failed to allocate 2405843009873693952 byte.

A known issue regarding such large request is solved with revision 50. In case this occurs in newer revisions, please contact SAP Support.

The following checks can then be done based on the information available so far:

No comments: