Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

I Lack of Fit in Ordinal Regression -- Analysis/Alternatives?

  1. Aug 25, 2016 #1

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Hi All,
    I ran a binary logistic of Y on three different numerical variables A,B,C respectively. I am having an issue of separation of variables with all of them, meaning that there are values Ao,Bo, Co for each of A,B,C (different values for each, of course) so that for ## A>Ao, B>Bo, C>Co ## all the responses are successes (I guess this forces the slope to diverge to minus infinity for the slope of the curve to accommodate the abrupt change of 1 to 0). Then I increased the success levels to three: high, medium and low, to use an ordinal regression . But now I have a significant lack of fit, with p -->0 on the Chi-squared test. How does one interpret lack-of-fit issues with a Logistic Regression? I know that a lack of fit in a simple linear means that data is not linear but what does it mean for a Logistic? Does it mean the (log of) the data is not distributed like an S-curve ExpL/(1+ExpL) (##L=
    \beta_0+ \beta_1 x+......##) ? If so, are there any standard , or any, alternatives (e.g for a distribution for the data). Any ideas?
     
  2. jcsd
  3. Aug 25, 2016 #2

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    What are your covariates? What is the nature of the covariates? Are they continuous? categorical?
     
  4. Aug 25, 2016 #3

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    They are all continuous, thanks.
     
  5. Aug 25, 2016 #4

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    That could be cause for problems in your hypothesis tests then. I don't know which test you used for lack of fit, but usually they don't work for continuous covariates.
     
  6. Aug 25, 2016 #5

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    No, I had no problem with the Chi-Squared, which AFAIK does not require discrete/categorical variables. I just got a pretty low p -value.
     
  7. Aug 25, 2016 #6

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    I don't understand. What is chi-squared? There are many chi-square tests in regression.
     
  8. Aug 25, 2016 #7

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    It is, I believe, the standard goodness of fit " [Sum(observed -expected)/observed]^2 I iam not aware of any other Chi-square goodness of fit tests.
     
  9. Aug 25, 2016 #8

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    Are you talking about the Pearson residuals? In either case, that chi-square test in your post doesn't always work for continuous variables.
     
  10. Aug 25, 2016 #9

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Thanks, I'll look into it.
     
  11. Aug 27, 2016 #10

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Still, it would be nice if someone knew of a good interpretation for a lack of fit in ordinal logistic, other than
    obvious ones on collinearity, etc. Lack of fit for ordinary least squares means a line is not an effective way of describing a dataset, but not so clear for logistic. I have broken down the process of linearity of log(odds) =
    ##\beta_0 + \beta_1x_1+... ## of how ##\beta_0## shifts the S-curve while ##\beta_1 ## "speeds it up or slows it down", etc. , but I am having trouble finding a clear understanding of the lack of fit.
     
  12. Aug 27, 2016 #11

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    Is this a proportional odds model?
     
  13. Aug 27, 2016 #12

    Stephen Tashi

    User Avatar
    Science Advisor

    It's easier to analyze real life situations as real life situations rather than mathematical skeletons. What phenomena does the data represent ?
     
  14. Aug 27, 2016 #13

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    EDIT2 I did a regress of control v compliance/ effectiveness. Specifically, control measures vs the existence of Fraud (F), Error (E) and Waste (W). A linear regression for each separately produces the expected results: increased control leads to a decrease of each of F,E,W. I was trying to do a Logit of Control vs each, to get a measure of proportionality to have some ideas of the odds of a certain level of control leading above or below a cutoff point ( selected as a standard level of 2.5 in a scale of 0 to 5 ) in each of the variables F,E,W. I got a horrible fit for binary regressions with the Chi-Squared and Pearson goodness of fit methods, with a P of 0.00 (Actually, I had a separation of points issue, since, beyond a certain level of control, all responses were successes). I tried using a Likert scale to change the binary into an ordinal logistic, see if I got a better fit , with no success EDIT( and low concordance, so low Kruskal's, etc.).
     
    Last edited: Aug 27, 2016
  15. Aug 27, 2016 #14

    Stephen Tashi

    User Avatar
    Science Advisor

    An elementary question: Is each sample datum defined by 4-tuple of numbers: ( c,f,e,w) so all four values apply to a single "situation" that provides one sample ?
     
  16. Aug 27, 2016 #15

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Yes, for a certain fixed level of control we evaluate the associated levels of fraud, error and waste.
     
  17. Aug 27, 2016 #16

    Stephen Tashi

    User Avatar
    Science Advisor

    That's a hard sentence to parse. For example, "odds of" and "probability of" have different meanings. It's easier for me to think about probability that odds.

    I don't understand what "proportionality" means in that context. I think of a "proportion" as a ratio of a part to a whole. So what quantity is the "the part" and what quantity is "the whole"?

    When you say "in each of the variables" , are you asking about all of them simultaneously? Or are you analyzing them individually ? For example if the level of control is (say) 8, are you asking something about the probability that a situation where the control is 8 will have less than a level of 2.5 in all three of F,E,W ?
     
  18. Aug 27, 2016 #17

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Hi, sorry for the mess, they were closing the coffee shop and I wrote things in a hurry/
    1) I meant probability. I am new to logistic regression. As I understand it (please correct me if I am wrong ) the input is a collection of Bernoulli trials ( or at least their
    outcomes) and the outcome is a smooth family of Bernoulli distributions obtained through the use of Max Likelihood Estimators for the collection of outcomes. In other words, our output is a PDF from the family of S -curves with parameters the dependent variables.

    2)Re proportionality, I was being loose again. I meant a PDF relates the dependent variable to the independent ones, assigning a probability to input values for each independent variable.

    3) Re " In each of the variables" . Both, linearly I regress C against each individually and then against all of them (I ultimately do a "best subsets" analysis. considering all possible combination of regressions, the best one being the one with lowest Mallows' Cp and highest adjusted R^2; in case of tie, select the model with the fewest variables. The 3-variable model was the best). I also regressed each independent variable (i.e., F,E,W) logistically against Control . But I don't know how to do a logistic regression in the opposite sense, i.e., to have a control input and get probabilities for each of the 3 variables.
     
  19. Aug 28, 2016 #18

    Stephen Tashi

    User Avatar
    Science Advisor

    Calling the outcome a "smooth family" of distributions is an interesting way to look at it. The outcome gives the parameter p of a Bernoulli distribution as a function of some independent variable x. The fitted equation p = f(x) implicitly defines a smooth family of Bernoulli distributions because for each given x we have a Bernoulli distribution with parameter p(x).


    A "family" of distributions defines more than one PDF. Each member of the family has a PDF.

    We can investigate that concept, but it seems "backwards" to how the usual sort of analysis goes. When I think of "fraud" and "control" (like frequent audits), I think of the a level of "control" causing (or allowing) a level of "fraud". I don't think of "fraud" as being what causes "control" (although I suppose one could look at it that way).

    That would say that C is the dependent variable, so for logistic regression it has 2 possible outcomes , which I'll call "low control" and "high control". Each of the variables F,E,W is regarded as having a continuous range of values. Is that correct ?
     
  20. Aug 28, 2016 #19

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Yes, I wrote the variable order backwards. I was hoping to actually do a multi-valued , i.e., control as independent and a triple (F,E,W) of values as functions of
    control, assigning a probability triple for fixed values. Obviously, these three, F,E,W depend on C and not vice-versa. Still, going back to the initial question: how do we interpret a lack-of-fit in this case ( or, better, in general)?
     
  21. Sep 10, 2016 #20

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    Just a followup: Say we are working on the same situation as above: we have a logistic of control (C) vs each of F,E,W (Fraud, Error, Waste)
    Say we also assume each of F,E,W to have the same importance. It seems to make sense to logistically regress (binary) C against the arithmetic average:

    D:= (F+E+W)/3 Any caveat to consider? I am trying to consider the cutoff point for this D to be the mean value of the means of each of F,E,W , i.e., this cutoff point determines a case (and anything below etermines a non-case; in the case of an equality we can randomly decide a yes or no.)
    Is this a meaningful way and a standard way of doing things?.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Lack of Fit in Ordinal Regression -- Analysis/Alternatives?
  1. Regression analysis? (Replies: 11)

Loading...