Unit 5 Assignment 1

docx

School

Capella University *

*We aren’t endorsed by this school

Course

PSY7610

Subject

Psychology

Date

Dec 6, 2023

Type

docx

Pages

Uploaded by SuperFieldFalcon19

Running head: EVAL TECHNICAL QUALITY 1 Evaluation of Technical Quality Unit 5 Assignment 1 PSY – 7610 Tests and Measurements Dr. Lynette Bujack Capella University Fall 2023 Lindsay Simmons

Running head: EVAL TECHNICAL QUALITY 2 Introduction For the previous assignment, the selected standardized tests consisted of the Assessment of Basic Language and Learning Skills, Revised (ABLSS-R), the Verbal Behavior Milestone and Placement Program (VB-MAPP), and Motivation Assessment Scale (MAS) of behavioral analytic skills assessments. These standardized tests are used to assess various skills of individuals in the educational system who struggle with a disability. However, one of the main mental disorders which are observed is known as Autism Spectrum Disorder (ASD). While there are various types of ASD, these tests may access the impacted individual’s behavioral, social, and verbal functions from childhood to adulthood (Lord et al., 2020). In Educational Psychology, disability counselors and professors in the higher educational level may require documentation to provide the necessary accommodations for disabled students to succeed as well as understanding from an applied behavioral analysis standpoint of view (Barnard-Brak et al., 2010). Technical Review ABLLS-R Usry, J., Partington, S. W., & Partington, J. W. (2018). Using expert panels to examine the content validity and inter-rater reliability of the ABLLS-R. Journal of Developmental and Physical Disabilities, 30(1), 27-38. For this study, the purpose of the research is to determine content validity and inter-rater reliability of the ABLSS-R by the collected data of 2 expert panel groups (Usry et al., 2018). The first expert panel was comprised of 6 certified behavioral analysts. Each of these analysts had a minimum of 5 years of experience in working with children formally diagnosed with ASD, or if the individual was uncertified at the time, at least 7 years of experience. Each expert in the panel group completed the required training on administration of the ABLSS-R. The second expert

Running head: EVAL TECHNICAL QUALITY 3 panel was comprised of 5 individuals without expert training, but possessed either 1 of 3 criteria, which were ASD treatment therapies, experience with ASD children in the educational environment, or a college degree with behavioral background. Both panels were allowed 3 weeks to complete the VB-MAPP assessment, and both panels met the allotted timeframe. The first panel yielded an 81% content validity with 441 out of 544 items, which were considered essential, while the second panel yielded an intraclass correlation coefficient of 0.95 and a p value less than 0.001 (Usry et al., 2018). The results of this study suggest that ABLSS-R produced valid results from the assessment procedures. Partington, J. W., Bailey, A., & Partington, S. W. (2018). A Pilot Study Examining the Test– Retest and Internal Consistency Reliability of the ABLLS-R. Journal of Psychoeducational Assessment, 36(4), 405–410. The purpose of this study is to find evidence of internal consistency and test-retest reliability for ABLLS-R due to the lack of empirical evidence using data from young children which are not characterized by the range of autistic behavior (Partington et al., 2018). The children in the study consisted of 52 individuals ranging from the ages of 6 to 72 months of age with a total of 50 undergraduate assessors, who received training on administering the ABLSS-R assessment. All assessors knew the child who was taking the ABLSS-R assessment in the home setting for familiarity and comfort purposes. All data was entered into the WebABLSS-R system online during a total of 3-month intervals. All children scored within average developmental ranges. The study yielded evidence of decent test-retest reliability with correlation coefficients of 0.84 at 3 months, 0.77 at 6 months, and 0.62 at 9 months, and 0.54 at 12 months. The study

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Running head: EVAL TECHNICAL QUALITY 4 yielded profound evidence of internal consistency reliability, considering the alpha coefficient was greater than 0.90 (Partington et al., 2018). Technical Review VB-MAPP Montallana, K. L., Gard, B. M., Lotfizadeh, A. D., & Poling, A. (2019). Inter-rater agreement for the milestones and barriers assessments of the verbal behavior milestones assessment and placement program (VB-MAPP). Journal of Autism and Developmental Disorders, 49(5), 2015-2023. Intraclass correlation coefficients were used to measure the VB-MAPP obstacles and milestone assessments in this study (Montallana et al., 2019). There were 32 participants diagnosed individuals with ASD who received a referral from a doctor for behavioral assessment services due to no experience with applied behavioral analysis therapy. A total of 24 experienced assessors, who received self-paced training by the manuals, administered the VB-MAPP in a familiar environment of the client. This environment was part of the client’s daily routine, whether it was the home, a facility, or a comfort area. These assessors also had experience with educational purposes in behavioral analysis. All data was collected and recorded on a scoring sheet, which displayed finalized results of a 98.7% reliable data input with reliable coefficients of 0.876 and 0.629 for the Barriers and Milestones Assessment (Montallana et al., 2019). A few noted limitations of this study may include a lack of age distribution, skewed gender results, expertise levels of the assessors, and the type of ASD, from mild to severe. Psychometric properties of the VB-MAPP are not established, and the assessment requires evaluation of reliability with additional studies and research (Montallana et al., 2019).

Running head: EVAL TECHNICAL QUALITY 5 Barnes, C. S., Mellor, J. R., & Rehfeldt, R. A. (2014). Implementing the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP): Teaching Assessment Techniques. The Analysis of verbal behavior, 30(1), 36–47. The evidence of validity in the VB-MAPP was conducted in this study by completing research on the effects of test assessor training pertaining to the milestone assessment (Barnes et al., 2014). Effective measurements from the VB-MAPP assessment must have assessors with prior knowledge and previous experience. It is helpful if the assessors have knowledge of Skinner’s analysis of verbal behavior, or else, then there could be a potential error in the data collection. The assessor could misinterpret the operant and the responses from the test takers, resulting in error variance, incorrect results, and skewed data interpretation. For this study, there are three milestones for the assessment, but only two of the milestones are the focus, being stages 1 and 2. The interobserver agreement data yielded results of 82.5% and 85.9% from the test administrators, which were 2 psychologists who presented the VB-MAPP to young children with a formal diagnosis of ASD. These 2 psychologists were instructed to follow the VB-MAPP manual and behavioral training, which covered the basis of instruction, the overview of the model, rehearsal of data, feedback from clients, and remedial if the assessor scored less than a 90% (Barnes et al., 2014). The first psychologist had an increase in score from a 64% and 51.9% to a 94.5% and 88.25%, while the second psychologist had an increase in score from 58.4% and 55.3% to a 91.6% and a 94.1%, in stages 1 and 2. Once more, additional research is required for more effective results in administration of the VB-MAPP to determine the psychometric dimensions of the assessment assessors (Barnes et al., 2014). Technical Review MAS

Running head: EVAL TECHNICAL QUALITY 6 Koritsas. (2013). Psychometric comparison of the Motivation Assessment Scale (MAS) and the Questions About Behavioral Function (QABF). Journal of Intellectual Disability Research., 57(8), 747–757. This study examined the psychometric comparison of MAS and Questions About Behavioral Function (QABF). In this comparison, there were several factors observed from adulthood individuals who had behavioral issues or an intellectual disability of the mind. The factors observed consisted of construct validity, convergent validity, internal consistency reliability, and inter-rater reliability (Kortitsas, 2013). A total of 70 adult participants were recruited through a disability service and assigned a support worker. These support workers had known the adult participant for a minimum of 6 months. A second support worker was also assigned to obtain the inter-rater reliability data for the study. The data yielded decent internal consistency results with an alpha coefficient above 0.70. The data yielded good intra-class correlation coefficient results with a score above 0.75, which in turn suggests strong inter-rater reliability data. The convergent validity and construct validity resulted in N=70, which suggests a strong correlation between the results. The conclusion of this study reported the MAS was reliable and consistent based upon the data (Kortitsas, 2013). Joosten, A. V., & Bundy, A. C. (2008). The motivation of stereotypic and repetitive behavior: Examination of construct validity of the motivation assessment scale. Journal of Autism and Developmental Disorders, 38(7), 1341-8. The purpose of this study was to examine the construct validity of the MAS assessment by comparing the differences in scores between two groups of children with disorders (Joosten &

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Running head: EVAL TECHNICAL QUALITY 7 Bundy, 2008). The differences in scores were examined between children with ASD and an intellectual disability, and children with just an intellectual disability. A total of 67 children participants between the ages of 5 years and 18 years were included in the study. These children received a formal diagnosis of an intellectual disability and attended a specialist school with experienced staff and instructors. The instructors completed the MAS assessment for each individual and observed the children over a 6-week period. The children with intellectual disabilities included a total of 114 observations. The children with both ASD and an intellectual disability included a total of 132 observations. The results failed to provide data to prove construct validity validation for the 2 groups due to differences in response motivation (Joosten & Bundy, 2008). Conclusion The conclusion of examining the research articles provided a lack of evidence for sound psychometric parameters for ABLLS-R, VB-MAPP, and MAS. These assessments are practiced frequently and widely used by educators, psychologists, and behavioral analysts. However, there is a dire need for more research to establish reliability and validity with ABLSS-R, VB-MAPP, and MAS for future purposes in observing behavioral disorders. ASD consists of broad behaviors depending on the mild to severe formal diagnosis, and therefore, require individualized tests based upon the type of autism. Considering these factors, then it is impossible to create a universally recognized testing assessment to cater to ASD for both reliability and validity content parameters.

Running head: EVAL TECHNICAL QUALITY 8 References Barnard-Brak, L., Lechtenberger, D., & Lan, W. Y. (2010). Accommodation strategies of college students with disabilities . The Qualitative Report , 15 (2), 411–429. 10.46743/2160-3715/2010.1158 Barnes, C. S., Mellor, J. R., & Rehfeldt, R. A. (2014). Implementing the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP): Teaching Assessment Techniques. The Analysis of verbal behavior, 30(1), 36–47. Joosten, A. V., & Bundy, A. C. (2008). The motivation of stereotypic and repetitive behavior: Examination of construct validity of the motivation assessment scale. Journal of Autism and Developmental Disorders, 38(7), 1341-8. Koritsas. (2013). Psychometric comparison of the Motivation Assessment Scale (MAS) and the Questions About Behavioral Function (QABF). Journal of Intellectual Disability Research., 57(8), 747–757. Lord, C., Brugha Traolach, S., Charman, T., Cusack, J., Guillaume, D., Frazier, T., Jones Emily, J. H., Jones, R. M., Pickles, A., State, M. W., Taylor, J. L., & Veenstra-VanderWeele Jeremy. (2020). Autism spectrum disorder (primer). Nature Reviews: Disease Primers, 6(1). Montallana, K. L., Gard, B. M., Lotfizadeh, A. D., & Poling, A. (2019). Inter-rater agreement for

Running head: EVAL TECHNICAL QUALITY 9 the milestones and barriers assessments of the verbal behavior milestones assessment and placement program (VB-MAPP). Journal of Autism and Developmental Disorders, 49(5), 2015-2023. Partington, J. W., Bailey, A., & Partington, S. W. (2018). A Pilot Study Examining the Test– Retest and Internal Consistency Reliability of the ABLLS-R. Journal of Psychoeducational Assessment, 36(4), 405–410. Usry, J., Partington, S. W., & Partington, J. W. (2018). Using expert panels to examine the content validity and inter-rater reliability of the ABLLS-R. Journal of Developmental and Physical Disabilities, 30(1), 27-38.

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Unit 5 Assignment 1

Related Documents