[#87] Allow for a sparse X matrix

Date:
2008-01-16 18:10 Priority:
3

State:
Open

Submitted by:
Douglas Bates (dmbates) Assigned to:
Douglas Bates (dmbates)

Product:
lme4 Operating System:
All

Component:
lmer

Summary:
Allow for a sparse X matrix

Detailed description
For some models it will be useful to allow for a sparse X matrix. We could allow the X slot to have length 0 and include a dgCMatrix object in a Xs slot. If X has nonzero length it is used, otherwise Xs is used. It would be necessary to change the logic of update_mu and update_RX. I'm not sure if other changes would be needed. One application for this would be for obtaining the "marginal maximum likelihood" estimates of IRT models.

Comments:

Message
Date: 2009-08-19 13:54 Sender: Douglas Bates This has now been implemented in the lme4a package under lmer2. Soon we will convert the general approach in lme4a to what is now called lmer2.
Date: 2008-01-17 18:19 Sender: Douglas Bates What I was proposing is simpler than allowing a class union. I would keep the X slot as a matrix with p columns and add an Xs slot as a dgCMatrix. If X has n rows it is used. If X has 0 rows then Xs is used. I am adopting a convention of having possibly redundant numeric slots that are of length 0 if they are not to be used. In lmer.c I define and use a macro SLOT_REAL_NULL that checks the length of a slot and returns the REAL part when the length is non-zero, otherwise it returns (double)NULL. That way you just retain the pointer and check the pointer itself before trying to extract the contents. For example, in many cases where there are optional weights that would default to unit weights you will see expressions like y[i] (wts ? wts[i] : 1)
Date: 2008-01-17 16:46 Sender: Martin Maechler IIUC, your proposal would have slot classes X = "dgeMatrix_or_NULL" Xs= "dgCMatrix_or_NULL" Alternatively, I'd propose to just have X = "dgeMatrix_or_dgCMatrix" (which we'd probably rather call "dgeneralMatrix" or "generaldMatrix" -- since we already have "generalMatrix" and "dMatrix"). In both proposals, you need formal or informal class unions and an "if(dense) .. else .." clause.

No Changes Have Been Made to This Item