about the PARAM_POSITION Dialog

Cast the long value you get to LPPOS.
You can find the definition in cncf.h.
Usually directly cast is enough, but for some cases (e.g., obstacle type) you need to get the value by cast param to eventParam struct and get the value in W or L, according to Yves's comment.

Edit: if your extension uses more than two params, then cast param1 or 2 is not enough, in this case you need to get the currentParam pointer.
The macro provided to read params will update this pointer, so be careful to not mess the order.

eventParam* param = rdPtr->rHo.hoCurrentParam;
param = (eventParam*)((BYTE*)(param) + param->evpSize * index);

This is possible for absolute positions only. Before doing CNC_GetParameter, do this:

LPPOS pPos = (LPPOS)&rdPtr->rHo.hoCurrentParam->evp.evpW.evpW0;

This gives you access to the PositionParam structure. In this structure, posLayer gives you the layer, but only if it's an absolute position, = when posOINUMParent == -1.

If posOINUMParent != -1, then you have to take the layer from the "parent" object. Finding what is the parent object depends on the current selection, this is complex and this function is not available to extensions.

Quote from prester

Yves saying what If posOINUMParent == -1, But i'm not know about what the case of posOINUMParent will get the value -1.

When using actual instead of relative to…
Actually you can get parent object’s pointer, but a bit complex: get oil offset by oi then access to this object’s objectInfoList, and you can get first selection & offset to next, then you can iterate it as a chained list. Also you need to handle the case the object you relative to is a qualifier, method is the same but first you need to use a qualifier to object list to get the object info list with this qualifier, and call the iterate function of single object.
You can find an object selection helper class by Anders Riggelsen (Andos) on forum, which helps a lot for beginners. I wrote a custom version based on it with the help of uppernate here: Please login to see this link.
Playing with selection is a inevitable part of fusion, enjoy :p

Did you try what I said above?

Before doing CNC_GetParameter, do this:

LPPOS pPos = (LPPOS)&rdPtr->rHo.hoCurrentParam->evp.evpW.evpW0;

If the user chose "Actual coordinates", then posOINUMParent = -1 and posLayer gives you the index of the layer selected in the drop down list.

if (pPos->posOINUMParent == -1)
{
    // Actual coordinates
   layer = pPos->posLayer;
}
else
{
    // Relative to
    // This case is more complex to handle as it depends on object selection AND if you are in a condition, action or expression...
}

I suggest that if you want a layer information in all cases you forget PARAM_POSITION and use your own X, Y and Layer parameters.