Announcement

Collapse

Civics 101 Guidelines

Want to argue about politics? Healthcare reform? Taxes? Governments? You've come to the right place!

Try to keep it civil though. The rules still apply here.
See more
See less

Ireland recovering from Theocracy.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by Sam View Post
    The four nationally representative data sets
    Indeed, I don't deny that there have been a few studies using more representative data sets, which needs to be look at in greater detail. What I did say (and what I just posted to Starlight about 60 seconds ago) was that most of the "dozens/50+/whatever" studies cited are not representative, yet they are still cited to boost the numbers for the express purpose of making it seem like there's an overwhelming case, which is utterly dishonest by the experts that knowingly do so, and a demonstration of gross incompetency otherwise.

    Originally posted by Paprika View Post
    Such research should be dismissed (as Regnerus and the others) or counted as very weak evidence, yet you and yours use the numbers to try and impress "Oh look we have many dozens of studies!"
    On a sidenote, could we please have the same discussion in only one thread?
    Last edited by Paprika; 05-27-2015, 01:11 AM.

    Comment


    • Originally posted by Adrift View Post




      Uggh. For the life of me, I can't understand why otherwise intelligent folk like yourself get suckered into type indicator tests like these. Its the modern day equivalent of a horoscope.

      When asked how scientifically valid the test was, here's was the decent breakdown of it's issues from one of Reddit's askscience subs.


      Source: http://www.reddit.com/r/askscience/comments/1p2cki/how_scientifically_valid_is_the_myers_briggs/

      I am the lead psychometrician at a personality test publisher, so I will attempt to answer your question.

      To begin, it is important to note that no test is "scientifically valid". Validity is not an element of a test, but specifically has to do with test score interpretation. (see the Standards for Educational and Psychological testing 1999, or Messick, 1989). That being said, the Myers Briggs is not a scientifically valid personality assessment. However, personality assessments can be validated for specific purposes.

      Moving onto the bigger issue with the Myers-Briggs: Decision consistency. The Myers-Briggs proclaims a reliability (calculated using coefficient alpha) of between .75-.85 on all of its scales (see Myers-Briggs testing manual). These are general, industry standard reliability coefficients(indicating that if you were to retest, you would get a similar score, but not exact). However, the Myers-Briggs makes additional claims about bucketing individuals into 1 of 16 possible personality types. That you can shift up or down a few points if you were to retake the test on any of the four distinct scales means that you may be higher on one scale than another simply through retaking the test due to measurement error. In fact, literature shows that your personality type will change for 50% of individuals simply through retesting. (Cautionary Comments Regarding the Myers-Brigg Type inventory, Consulting Psychology Journal: Practice and research, summer, 2005). This result indicates very low decision consistency. The low decision consistency is also a mathematical inevitability given 16 personality profiles using 4 scales and scale reliability around .8.

      Given the low decision consistency, and given that claims the Myers-Briggs makes about about your personality(validity information) depends on the decisions made by the test to be consistent and not subject to change simply based on retesting, it is highly unlikely that there can be a solid validity argument supporting the Myers-Briggs as a personality indicator. Maybe there are studies showing that it can be used in a very specific context, but sweeping generalizations about the tests use are not going carry much weight.

      Now, as a working professional in the field, the Myers-Briggs does NOT have a good reputation as being a decent assessment. It has marketed well to school systems and has good name recognizability, but it is not a well developed exam. There are much better personality assessments available, such as SHL's OPQ32 or The Hogan personality inventory. Now, I don't want to say any of these are good. The best correlations between job performance and personality assessments is about .3 (indicating about 9% of the variance in a persons job performance can be accounted for by a personality assessment). That is the BEST personality assessments can do in terms of job performance... and a correlation of .3 is not worth very much (considering that tests like ACT or the SAT can correlate upwards of .7 with first year college GPA under ideal circumstances).

      Expanding on this, the Myers-Brigg's is not only psychometrically unreliable, it is neither a psychometrically valid nor a theoretically validated assessment of personality. It posits a very distinct structure of personality. We know from Popper's (1934) original argument that the more specific a hypothesis, the easier it is to falsify. This is very much so in Myers-Brigg's case. The process in validating an assessment includes a number of statistical and methodological techniques that include assessing construct, content, discriminant, and convergent validities. Below are several links that reveal the shortcomings in the Myers-Brigg's in attempting to achieve this level of psychometric validity:

      * Factor analysis procedures are not consistent, questioning it's construct validity

      * Concerns about the utility of the test, more validity problems

      * Item construction issues

      * Doesn't map onto the Big 5, an extremely well-validated personality assessment

      * "Routine use of the MBTI is not recommended" More validity and reliability issues.

      I was actually surprised at how difficult it was to find any psychometic testing on the MBTI. The reason being that academia has long since abandoned it for other better assessments.

      © Copyright Original Source


      MBTI isn't useful as a scientifically rigorous evaluation tool. It is, however, highly useful (and pretty in line with) other basic personality type guidelines. So long as someone doesn't imagine that people fall neatly and absolutely into a type or that an understanding of personality types and trends equates to an understanding of a personality, these things are highly useful -- especially for folks like me who do not intuit social and personal cues.

      Using them as guidelines, the types work very well for me. As with all pseudoscience, YMMV.
      "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

      Comment


      • Originally posted by Sam View Post
        Regarding sexual development, I would include measures pertaining to sexual activity or sexual risk-taking but not measures pertaining to sexual orientation or tolerance of non-normative sexual preferences. On the broader question, I think the abstract I cited listed the primary measures of development:

        Source: Ibid



        We conclude that there is a clear consensus in the social science literature indicating that American children living within same-sex parent households fare just, as well as those children residing within different-sex parent households over a wide array of well-being measures: academic performance, cognitive development, social development, psychological health, early sexual activity, and substance abuse. Our assessment of the literature is based on credible and methodologically sound studies that compare well-being outcomes of children residing within same-sex and different-sex parent families.

        © Copyright Original Source



        Emphasis added.
        So, just so I have this right, you would agree that the papers I pointed out do, in fact, relate to at least a few of the primary measures you're concerned with, correct? (social development, psychological health, and early sexual activity).

        Comment


        • Originally posted by Sam View Post
          MBTI isn't useful as a scientifically rigorous evaluation tool. It is, however, highly useful (and pretty in line with) other basic personality type guidelines. So long as someone doesn't imagine that people fall neatly and absolutely into a type or that an understanding of personality types and trends equates to an understanding of a personality, these things are highly useful -- especially for folks like me who do not intuit social and personal cues.

          Using them as guidelines, the types work very well for me. As with all pseudoscience, YMMV.
          Well, as long as you recognize it for what it is.

          Comment


          • Originally posted by Paprika View Post
            Indeed, I don't deny that there have been a few studies using more representative data sets, which needs to be look at in greater detail. What I did say (and what I just posted to Starlight about 60 seconds ago) was that most of the "dozens/50+/whatever" studies cited are not representative, yet they are still cited to boost the numbers for the express purpose of making it seem like there's an overwhelming case.

            So far, inasmuch as I know, no "gold standard" meta-reviews have contradicted the consensus that children of same-sex partners are developmentally on-par with children of opposite-sex partners. Studies based on NLLFS or other weaker datasets are not worthless, like Regnerus' study, and need not be dismissed. While one may well dispute the strength of such studies, the burden is on them to provide studies of higher quality that contradict both these weaker studies and the "gold-standard" studies. Otherwise, it's (Side A: strong+weak evidence) vs. (Side B: weak evidence).


            Originally posted by Paprika View Post
            On a sidenote, could we please have the same discussion in only one thread?
            Oh, if only. Herding chickens, man.
            "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

            Comment


            • Originally posted by Adrift View Post
              So, just so I have this right, you would agree that the papers I pointed out do, in fact, relate to at least a few of the primary measures you're concerned with, correct? (social development, psychological health, and early sexual activity).
              I would agree that those things are primary measure of childhood and adolescent development, yes. I have to be honest and admit I haven't looked at all of them (OK, just one abstract so far) because I'm multitasking (distracting myself) here and legal work on the other monitor. I will get to checking 'em out tomorrow, though.
              "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

              Comment


              • Originally posted by Adrift View Post
                Well, as long as you recognize it for what it is.
                Yup. No illusions — it's a pragmatic tool just one level higher than personal experience. Gets the job done.
                "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

                Comment


                • Originally posted by Sam View Post
                  So far, inasmuch as I know, no "gold standard" meta-reviews have contradicted the consensus that children of same-sex partners are developmentally on-par with children of opposite-sex partners.
                  The 'consensus' was precisely founded on the dozens of non-representative (eg snowball sampled) studies, mostly with small sample sizes, that make an invalid comparison between better-off homosexual marriages to the average heterosexual ones, effectively comparing apples to oranges. Ie. the 'consensus' was bunkum.

                  Studies based on NLLFS or other weaker datasets are not worthless, like Regnerus' study, and need not be dismissed.
                  Excellent to hear you say that, because there are those who would dismiss it entirely.

                  Otherwise, it's (Side A: strong+weak evidence) vs. (Side B: weak evidence).
                  Right, let's look at the "strong" evidence. Which of the four studies (that you mentioned used nationally representative data) should we start with?

                  Comment


                  • Originally posted by Paprika View Post
                    The 'consensus' was precisely founded on the dozens of non-representative (eg snowball sampled) studies, mostly with small sample sizes, that make an invalid comparison between better-off homosexual marriages to the average heterosexual ones, effectively comparing apples to oranges.


                    Excellent to hear you say that, because there are those who would dismiss it entirely.


                    Right, let's look at the "strong" evidence. Which of the four studies (that you mentioned used nationally representative data) should we start with?
                    The studies I remember going over here years ago used the NLLFS but paired the data to similar samples in the general population, thereby effectively comparing apples to apples. So it would be wrong to assume that those dozens of studies are not meritorious.

                    An in-depth review of any particular study is going to have to wait till at least tomorrow, I'm afraid. Feel free to grab one of the "strong" studies and identify any critiques you have and/or reasons to doubt the conclusion. I'll try to jump back in head-first tomorrow. Assuming I can break my current writer's block on this legal thing.
                    "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

                    Comment


                    • Oh, in an earlier post I made an addition (in bold) to my post that unfortunately wasn't made in time for you to reply to earlier; I was hoping you would comment on it.

                      Originally posted by Paprika View Post
                      Indeed, I don't deny that there have been a few studies using more representative data sets, which needs to be look at in greater detail. What I did say (and what I just posted to Starlight about 60 seconds ago) was that most of the "dozens/50+/whatever" studies cited are not representative, yet they are still cited to boost the numbers for the express purpose of making it seem like there's an overwhelming case, which is utterly dishonest by the experts that knowingly do so, and a demonstration of gross incompetency otherwise.

                      Comment


                      • Originally posted by Sam View Post
                        The studies I remember going over here years ago used the NLLFS but paired the data to similar samples in the general population, thereby effectively comparing apples to apples. So it would be wrong to assume that those dozens of studies are not meritorious.
                        1. The NLLFS is not representative as your own source notes:

                        convenience or snowball samples are more common in the literature, and the most widely used data source is the National Longitudinal Lesbian Family Study (NLLFS). The NLLFS is based on interviews with donor-inseminated lesbian mothers five times from insemination or pregnancy to the child’s 17th birthday (e.g., Gartell and Bos 2010; Goldberg et al. 2011; van Gelderen et al. 2012a) and since 2002, 15 studies used these data...Relying on convenience samples means that the same-sex parents within these studies are not representative of all same-sex parents and represent only those who were targeted and agreed to participate, perhaps selective of the most highly functioning families.


                        2. I'm not saying that the studies are useless, merely that the conclusion that some experts have drawn from them - the so-called 'consensus' - is bunkum.

                        An in-depth review of any particular study is going to have to wait till at least tomorrow, I'm afraid. Feel free to grab one of the "strong" studies and identify any critiques you have and/or reasons to doubt the conclusion. I'll try to jump back in head-first tomorrow. Assuming I can break my current writer's block on this legal thing.
                        Will do. I have things to do too but I'll try to get back as soon as possible.

                        Comment


                        • Originally posted by Paprika View Post
                          Oh, in an earlier post I made an addition (in bold) to my post that unfortunately wasn't made in time for you to reply to earlier; I was hoping you would comment on it.
                          I don't think that's a fair assessment, as such "weak" studies still have a lot of explanatory merit and authors (at least in the studies I've read) make efforts to mitigate the selection problems associated with using NLLFS. Even if such studies are weaker than gold-standard, there remains a significant difference between studies (strong+weak) showing developmental parity and studies (any strong?+weak) showing a lack of parity.

                          So it's not, in my opinion, unethical or wrong to say "The consensus shows" or even "The clear consensus is" — should strong (or even more weak) studies refute that consensus of extant studies, such statements would be wrong and unethical. As things stand today, I do not believe they are.
                          "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

                          Comment


                          • Originally posted by Paprika View Post
                            1. The NLLFS is not representative as your own source notes:

                            convenience or snowball samples are more common in the literature, and the most widely used data source is the National Longitudinal Lesbian Family Study (NLLFS). The NLLFS is based on interviews with donor-inseminated lesbian mothers five times from insemination or pregnancy to the child’s 17th birthday (e.g., Gartell and Bos 2010; Goldberg et al. 2011; van Gelderen et al. 2012a) and since 2002, 15 studies used these data...Relying on convenience samples means that the same-sex parents within these studies are not representative of all same-sex parents and represent only those who were targeted and agreed to participate, perhaps selective of the most highly functioning families.


                            2. I'm not saying that the studies are useless, merely that the conclusion that some experts have drawn from them - the so-called 'consensus' - is bunkum.


                            Will do. I have things to do too but I'll try to get back as soon as possible.
                            The NLLFS is not representative, no, but the studies I read (at least some of them, it's been some time) paired samples from the NLLFS with similar samples (location, age of partners, economic earning, etc.) from the general population. Thus, it could be determined whether there is an disparity between children of same-sex partners and children of opposite-sex partners that is inherent in the sexual identity of those partners.

                            As some have suggested that there is an inherent detriment to having same-sex parents, this is an important area of study.
                            "I wonder about the trees. / Why do we wish to bear / Forever the noise of these / More than another noise / So close to our dwelling place?" — Robert Frost, "The Sound of Trees"

                            Comment


                            • Originally posted by Sam View Post
                              I don't think that's a fair assessment, as such "weak" studies still have a lot of explanatory merit and authors (at least in the studies I've read) make efforts to mitigate the selection problems associated with using NLLFS. Even if such studies are weaker than gold-standard, there remains a significant difference between studies (strong+weak) showing developmental parity and studies (any strong?+weak) showing a lack of parity.

                              So it's not, in my opinion, unethical or wrong to say "The consensus shows" or even "The clear consensus is" — should strong (or even more weak) studies refute that consensus of extant studies, such statements would be wrong and unethical. As things stand today, I do not believe they are.
                              It is entirely dishonest to say of opposing studies that "we discuss the handful of recent studies reporting that children fare worse on any measure of child well-being (Allen et al. 2013; Goldberg et al. 2011; Gartrell et al. 2011; Regnerus 2012a, b), and each has shortcomings making broad generalizations impossible" while not noting that the same applies to most of the studies cited explicitly or implicitly ("consensus", "40+ studies").

                              Comment


                              • Originally posted by Sam View Post
                                The NLLFS is not representative, no, but the studies I read (at least some of them, it's been some time) paired samples from the NLLFS with similar samples (location, age of partners, economic earning, etc.) from the general population. Thus, it could be determined whether there is an disparity between children of same-sex partners and children of opposite-sex partners that is inherent in the sexual identity of those partners.
                                Ah, okay, I misread your earlier statement. I would, however, like to know which of these are the case.

                                On the other hand we are, I hope, agreed that studies that use sampling from the NLLFS but make the invalid comparison to the national average have little to no evidentiary weight and that any consensus founded on these is bunkum?
                                Last edited by Paprika; 05-27-2015, 01:41 AM.

                                Comment

                                Related Threads

                                Collapse

                                Topics Statistics Last Post
                                Started by little_monkey, Yesterday, 04:19 PM
                                6 responses
                                45 views
                                0 likes
                                Last Post whag
                                by whag
                                 
                                Started by whag, 03-26-2024, 04:38 PM
                                42 responses
                                230 views
                                0 likes
                                Last Post whag
                                by whag
                                 
                                Started by rogue06, 03-26-2024, 11:45 AM
                                24 responses
                                104 views
                                0 likes
                                Last Post Ronson
                                by Ronson
                                 
                                Started by Hypatia_Alexandria, 03-26-2024, 09:21 AM
                                32 responses
                                173 views
                                0 likes
                                Last Post Hypatia_Alexandria  
                                Started by Hypatia_Alexandria, 03-26-2024, 08:34 AM
                                73 responses
                                285 views
                                0 likes
                                Last Post Hypatia_Alexandria  
                                Working...
                                X