Re: How to check amount of data written on a preallocated device.

I haven’t seen what the output looks like for preallocated devices in V3, but try:

symcfg list -sid xx -pools -thin -v -detail

Then look at the section “other thin devices with allocations in this pool” It has for each device “Total Tracks”, “pool Allocated tracks” and “pool used tracks”

BTW, for VMAX 3, is there a particular reason you are pre-allocating? Certainly prior to V3, there was a little latency overhead for allocating tracks on first write to the newly allocated space, but much improved in V3, so this should not be a consideration now.

Related:

How to check amount of data written on a preallocated device.

I haven’t seen what the output looks like for preallocated devices in V3, but try:

symcfg list -sid xx -pools -thin -v -detail

Then look at the section “other thin devices with allocations in this pool” It has for each device “Total Tracks”, “pool Allocated tracks” and “pool used tracks”

BTW, for VMAX 3, is there a particular reason you are pre-allocating? Certainly prior to V3, there was a little latency overhead for allocating tracks on first write to the newly allocated space, but much improved in V3, so this should not be a consideration now.

Related:

3113982: Diagnosing Memory Heap Corruption in glibc with MALLOC_CHECK_

This document (3113982) is provided subject to the disclaimer at the end of this document.

Environment

Novell eDirectory 8.7.3 for Linux
Novell eDirectory 8.8 for Linux
Novell Open Enterprise Server (Linux based)
Novell SUSE Linux Enterprise Server 9
Novell SUSE Linux Enterprise Server 10
Novell SUSE Linux Enterprise Server 11

Situation

The GNU C Library (glibc) is the standard C library on Linux Systems. Other C libraries exist and are sometimes used for special purposes (such as very small subsets of the standard C libraries used for embedded systems and bootstrapping), but glibc is the standard C library on all Linux distributions.
The glibc includes three simple memory-checking tools. The two, mcheck() and MALLOC_CHECK_, enforce heap data structure consistency checking, and the third, mtrace(), traces memory allocation and deallocation for later processing. When memory is allocated from the heap, the memory management functions need someplace to store information about the allocations. That place is the heap itself; this means that the heap is composed of alternating areas of memory that are used by the program and by the memory management functions themselves. This means that buffer overflows or underruns can actually damage the data structures that the memory management functions use to keep track of what memory has been allocated.
When this happens, all bets are off, except that is pretty good bet that the memory management functions will eventually cause the program to crash. If we set the MALLOC_CHECK_ environment variable, a different and somewhat slower set of memory management functions is used. These functions are more tolerant of errors and can check for calling free() more than once on the same pointer for single-byte buffer overflows.

Resolution

Setting MALLOC_CHECK_ :

  1. If MALLOC_CHECK_ is set to 0 (zero), the memory management functions are simply most tolerant of errors and do not give warnings.
    • Maybe be useful if we are prevented from finding one memory bug by another that is not convenient to fix at the moment; it might allow us to use other tools to chase down the other memory bug.
    • It may also be useful if you are running code that works on another system but not on Linux. It can provide a quick workaround that may allow the code to temporarily function, before you have the chance to resolve the error.

  2. If MALLOC_CHECK_ is set to 1 (one), the memory management functions print out warning messages on standard error when problems are noticed.
    • It is useful if we are not aware of any problems and just want to be notified if any problem exist.

  3. If MALLOC_CHECK_ is set to 2 (two), the memory management functions call abort()when problems are noticed.
    • This is most useful from inside the debugger or a shell starting an application or daemon. It allows a backtrace to be obtained as soon as the memory management functions discover an error, providing information closest to the point at which the error has happened.
    • If a core is caused by a memory corruption, we have more information about memory allocations. This is better for troubleshooting and determining where/which application overwrote a memory address.

  4. Settings 1 and 2 can be combined by setting MALLOC_CHECK_ to 3 (three).

    This will enable the print out of warning messages on standard error (1), and will call abort() when problems are noticed (2).

Additional Information

Setting MALLOC_CHECK_ examples:
For ndsd/eDirectory:
  1. Add the following to the /opt/novell/eDirectory/sbin/pre_ndsd_start file:

    export MALLOC_CHECK_=3

    Note: the pre_ndsd_start script/file is usually empty. If there are other lines in the file, this should go first.

  2. Restart ndsd (/etc/init.d/ndsd restart) to activate.

NOTES:

– SLES 12 and RHEL 7

You must add all the environment variables required for eDirectory service in the env file located in the /etc/opt/novell/eDirectory/conf directory due to systemd requirements.
– This setting should be used for debugging purposes only.
Due to potential performance impact (which could be up to 25% in some cases), it is recommended to comment it out (and restart the process) as soon as the needed core file(s) are obtained.
For namcd:
  1. Add MALLOC_CHECK_=3 to NAMCD script (/etc/init.d/namcd):

    The script will then look similar to:



    # Template configuration variables

    prefix=/usr

    exec_prefix=/usr

    sbindir=/usr/sbin

    bindir=/usr/bin

    initdir=/etc/init.d

    export MALLOC_CHECK_=3

    userID=`id`

  2. Restart NAMCD (/etc/init.d/namcd restart) to activate

NOTE: This setting should be used for debugging purposes only.

Due to potential performance impact (which could be up to 25% in some cases), it is recommended to comment it out (and restart the process) as soon as the needed core file(s) are obtained.

Change Log

2017-Nov-17 – kklemm:

  • improved readability of the last part of situation.
  • updated additional info section to improve readability and consistency of potential performance impact.
  • changed from MALLOC_CHECK_=2 to MALLOC_CHECK_=3 per conversation with Darren McGary.
  • updated resolution section to enhance readability and understanding.

Disclaimer

This Support Knowledgebase provides a valuable tool for NetIQ/Novell/SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented “AS IS” WITHOUT WARRANTY OF ANY KIND.

Related:

7020973: SUSE products and a new security bug class referred to as “Stack Clash”.

A new class of vulnerabilitieshave been identified under the umbrella name “StackSmashing”.

This bug class exploits a weakness in theaddress space model of operating systems like Linux.

How does itwork…

The programs in operatingsystems use a so called stack for storing variables and returnaddresses used in functions. The stack grows depending on the amountof variables used and the depth of the called function tree. Thegrowth direction is also special, on most platforms it growsdownwards.

As the stack shares the same address space with theregular program, heap and libraries and other program memory regionscare needs to be taken that the automatic growing stack does notcollide with other memory regions.

For this some years ago a”stack guard gap” page of 4KB was introduced, that is alsoused for automatic growing the stack if a stack memory access goesinto the guard page.

The security research company Qualys hasidentified that in some libraries and programs under specificconditions the stack pointer can “jump over” this 4KB stackguard page and proceed below it or even overwrite memory areaspositioned there.

This can for happen with large arrays on thestack over 4KB which are accessed only in some places, or by programsusing the alloca() function to getstack memory that is also not accessed fully.

This grown stackcould then be made to “smash” into other memory areas,containing code, data, function pointers or similar and which in turncould be used to execute code.

Note that these problems arenot bugs in the programs, libraries or the kernel themselves, butcaused by vague interpretation of the stack grow magic ABI betweenthe compiler and kernel.

To mitigatethis class of attacks we will be doing the following :

– LinuxKernels are being released immediately.

The kernel updates willincrease the stack gap size to be much larger (1 MB / 16 MB),which should mitigate most of the cases found during research.

Thismitigation is tracked under CVE-2017-1000364

– glibcpackages are being released immediately.

glibc itself contains severalcases of being able to effect these stack jumps, happening evenbefore a binary is loaded in the dynamic loader.

When used withsetuid root binaries these could be used to escalate privilege fromuser to root using stack smashing.

This security fix is tracked underCVE-2017-1000366

– gcc (GNUCompiler Collection) updates will be released in the near future.

These updates will feature aflag that enables touching all stack memory pages when dynamic largestack allocations are done, to avoid having large jumps.

Note thatas the stack code is directly built into the libraries and binaries, recompiling packages is necessary to make it effective.

– Variousapplications might be updated in the near future.

We will identify and releaseupdates for various applications that have such stack usage patternsand rebuild them with the new gcc compiler flag.

Related:

7020973: SUSE products and a new security bug class referred to as “Stack Smashing”.

A new class of vulnerabilitieshave been identified under the umbrella name “StackSmashing”.

This bug class exploits a weakness in theaddress space model of operating systems like Linux.

How does itwork…

The programs in operatingsystems use a so called stack for storing variables and returnaddresses used in functions. The stack grows depending on the amountof variables used and the depth of the called function tree. Thegrowth direction is also special, on most platforms it growsdownwards.

As the stack shares the same address space with theregular program, heap and libraries and other program memory regionscare needs to be taken that the automatic growing stack does notcollide with other memory regions.

For this some years ago a”stack guard gap” page of 4KB was introduced, that is alsoused for automatic growing the stack if a stack memory access goesinto the guard page.

The security research company Qualys hasidentified that in some libraries and programs under specificconditions the stack pointer can “jump over” this 4KB stackguard page and proceed below it or even overwrite memory areaspositioned there.

This can for happen with large arrays on thestack over 4KB which are accessed only in some places, or by programsusing the alloca() function to getstack memory that is also not accessed fully.

This grown stackcould then be made to “smash” into other memory areas,containing code, data, function pointers or similar and which in turncould be used to execute code.

Note that these problems arenot bugs in the programs, libraries or the kernel themselves, butcaused by vague interpretation of the stack grow magic ABI betweenthe compiler and kernel.

To mitigatethis class of attacks we will be doing the following :

– LinuxKernels are being released immediately.

The kernel updates willincrease the stack gap size to be much larger (1 MB / 16 MB),which should mitigate most of the cases found during research.

Thismitigation is tracked under CVE-2017-1000364

– glibcpackages are being released immediately.

glibc itself contains severalcases of being able to effect these stack jumps, happening evenbefore a binary is loaded in the dynamic loader.

When used withsetuid root binaries these could be used to escalate privilege fromuser to root using stack smashing.

This security fix is tracked underCVE-2017-1000366

– gcc (GNUCompiler Collection) updates will be released in the near future.

These updates will feature aflag that enables touching all stack memory pages when dynamic largestack allocations are done, to avoid having large jumps.

Note thatas the stack code is directly built into the libraries and binaries, recompiling packages is necessary to make it effective.

– Variousapplications might be updated in the near future.

We will identify and releaseupdates for various applications that have such stack usage patternsand rebuild them with the new gcc compiler flag.

Related:

why does tacmd executecommand fail after upgrading to 6.3.0 FP07

After upgrading to 6.3.0. FP07 from FP06, tacmd executecommmand fails on Linux with this error
*** glibc detected *** /opt/IBM/ITM/lx8266/ue/bin/tacmd: double free or
corruption (!prev): 0x0000000000605bd0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x787f8)[0x7fe76ffe17f8]
/lib64/libc.so.6(cfree+0x6c)[0x7fe76ffe686c]
/opt/IBM/ITM/lx8266/ue/lib/executecommand(performexecutecommand+0x90c)[0
x7fe76ec01648]
/opt/IBM/ITM/lx8266/ue/lib/executecommand(executecommand+0x279)[0x7fe76e
bffde5]
/opt/IBM/ITM/lx8266/ue/bin/tacmd(main+0x19b8)[0x412856]
/lib64/libc.so.6(__libc_start_main+0xe6)[0x7fe76ff87c26]
/opt/IBM/ITM/lx8266/ue/bin/tacmd(__gxx_personality_v0+0xe2)[0x410d7a]

Related:

Network Information Service is exiting: Memory Access Error

Details
Product: Windows Operating System
Event ID: 8195
Source: Nissvc
Version: 5.2.3790.1830
Message: Network Information Service is exiting: Memory Access Error
   
Explanation

Network Information Service (NIS) is closing due to a memory allocation error. Client requests will fail.

Cause

Event 8195:  Memory allocation has failed in Server for NIS.

   
User Action

Event 8195:  Memory allocation has failed in Server for NIS. This can occur if the Windows Server operating system has insufficient virtual memory. Close idle applications to free as much virtual memory as possible, and contact Support Options from Microsoft Services (http://go.microsoft.com/fwlink/?LinkId=52267) if reducing the amount of virtual memory in use fails to correct the problem.

Related: