{"id":185643,"date":"2023-02-28T13:00:06","date_gmt":"2023-02-28T18:00:06","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=185643"},"modified":"2023-03-13T12:45:53","modified_gmt":"2023-03-13T16:45:53","slug":"nyed-data-explorer-shows-15-years-of-charter-school-success","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/","title":{"rendered":"NYED Data Explorer Shows 15 Years of Charter School Success"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>The article &#8220;NYED Data Explorer Shows 15 Years of Charter School Success&#8221; was originally posted on <a href=\"https:\/\/redwallanalytics.com\/2023\/02\/22\/nyed-data-explorer-shows-15-years-of-charter-school-success\/\">Redwall Analytics<\/a> Blog.<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"490\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-1100x490.png\" alt=\"\" class=\"wp-image-185682 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-1100x490.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-700x312.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-300x134.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-768x342.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics-1536x684.png 1536w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_app_all_students-rewall-analytics.png 2048w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/490;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: NYED Data<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">NYED Data Explorer filtered for \u201cAll Students\u201d ELA Aggregated Annual Test Scores<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-introduction\">Introduction<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote&nbsp;<a href=\"https:\/\/redwallanalytics.com\/2019\/11\/02\/how-does-stamford-charter-school-for-excellence-do-it\/\">How Does Stamford Charter School for Excellence do it?<\/a>. \u201cStamford Excellence\u201d was achieving remarkable results in a district with proficiency below 50%. Then and now, the school receives little recognition, and their effort to open a second Connecticut location in Norwalk, has been stymied as of this writing. My interest in these exceptional schools, and possibly the blog post, led to a new position as Director of Data &amp; Research at another high-performing Bronx charter school network,&nbsp;<a href=\"https:\/\/classicalcharterschools.org\/\">South Bronx Classical Charter Schools<\/a>. Naturally, when I discovered 15 years of&nbsp;<a href=\"https:\/\/data.nysed.gov\/\">NYED assessment data<\/a>, the interest to clean and free this data for others to discover in a Shiny app, was immediate. The opportunity to also feature Classical\u2019s stand-out performance didn\u2019t hurt my motivation, although this post and the app were built in my spare time, and do not represent the opinions of South Bronx Classical Charter Schools. Unlike many past Redwall posts, this one will not have code, and will be primarily to show how to use the app, built with&nbsp;<code>{golem}<\/code>&nbsp;and&nbsp;<code>{shinyWidgets}<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-classical-charter-schools-and-the-charter-debate\">Classical Charter Schools and the Charter Debate<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Classical Charter Schools operates four schools demonstrating outstanding academic performance in an under-served community, where there is not much evidence of educational accountability (based on the results which will be shown below). With approximately 90% economically-disadvantaged students, it achieves close to 90% proficiency in both ELA and Math, on par with the wealthiest NYS districts, while spending much less per pupil than DOE schools. This is accomplished with a variety of strategies, but most importantly for me, a big commitment to collecting and using data to make decisions. This post will show the result of extensive data cleaning, of over 3 million rows of school performance data, from the&nbsp;<a href=\"https:\/\/data.nysed.gov\/\">NYED Data<\/a>&nbsp;site. Despite five year\u2019s of daily experience manipulating data with code, untangling this data took much more effort than I care to admit. After the recent broad drop in proficiency related to the COVID, and the renewal of public debate about lifting the cap on new charter schools, it seems like there has never been a better time to make clean and accessible school performance data available. This post launches a \u201cminimum viable\u201d version of a Shiny App called&nbsp;<a href=\"https:\/\/luceyda.shinyapps.io\/nyedapp\/\">NYED Data Explorer<\/a>&nbsp;(accessible in this blog post below), which I hope may put to rest any questions over whether more charters should be allowed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-thoughts-on-nyed-and-open-public-data\">Thoughts on NYED and Open Public Data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">As with every public data site I have worked with, the NYED\u2019s disclosure strategy sometimes feels like it intends to make it hard to access a clean longitudinal data set. Disclosures occur annually, and are often in the inconvenient form of a MS Access Database, while other years shift to csv, xlsx or tab separated formats. Sometimes the Access databases include tables for enrollment, but leave them out in other years. In some years, only the current year is disclosed, and others also include the prior year. I was able to extract and separate these painfully, year-by-year from the command line using&nbsp;<code>mdbtools<\/code>. When the number of scholars in a group is below a threshold, fields are suppressed, which is common, but the NYED data denotes these cases with varying notations in different years. Sometimes, assessment data is disclosed in tables by grade, and others, all grades are stacked together in one table. Disclosed subgroups are usually male\/female and ethnicity, but several years include ethnic group by gender. Important fields are added, and then disappear, such as in 2022, dropping \u201cMean Scale Score\u201d, which showed the average score of a cohort at a school in most of the past years. As a result, I will concentrate on the \u201cPass Rate\u201d for now, which is the number of students scoring in Level 3 or 4 divided by total test takers. The cutoffs for levels change every year as does the test difficulty, so \u201cpass rates\u201d are also not an objective measure over time.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-data-collection-considerations\">Data Collection Considerations<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Schools record and report student attributes, but in many cases, it seems likely that these have differences in interpretation, inaccuracies in collection and timing differences. Tracking race and ethnicity accurately within a single institution with changing personal is likely to be complicated, much less across 4,000+ schools, all with varying, often manual processes. There may be dozens of data filings a school has to make over the calendar year to local DOE as well as the State systems. Even if a scholar is classified correctly, there will surely be many cases where the totals may not be added up or transposed correctly into reporting systems. Enrollment data changes through the year, but is primarily recorded at \u201cBEDS\u201d Day. There are many cases, especially before 2012, where the number of test takers at the exams in the Spring exceeds the total enrollment in a grade reported at the same school. In fact, in the early years of this data, the aggregate number of test takers often exceeded the total enrollment in all schools in a subgroup. This may be still an error in my data cleaning, but is unclear as of this writing. The intention of this app is to give the most accurate representation possible. Please understand that this is impossible given the nature of the data, although it lot better than anything else I have discovered up until now.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-navigating-the-app\">Navigating the App<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The&nbsp;<code>{golem}<\/code>&nbsp;app (shown below) was built with the&nbsp;<code>{shinyWidgets}<\/code>&nbsp;package, which offers \u201cconditional drop-down filtering\u201d through its&nbsp;<code>selectizeGroup<\/code>&nbsp;module, for navigating multiple categories with high-cardinality. By reducing the subset of each filter with the other filters, this method helps to rapidly narrow down to desired groups with little or no latency. I discovered this by accident after trying a lot of things which didn\u2019t work nearly as well, and still feel that it may not be well known as it deserves to be (see discussion in&nbsp;<a href=\"https:\/\/redwallanalytics.com\/2021\/08\/06\/introducing-the-redwall-irs-soi-tax-dashboard\/\">Introducing the Redwall IRS SOI Tax Dashboard<\/a>) discussed it in more detail.<\/p>\n\n\n\n<p class=\"has-text-align-center has-light-green-cyan-background-color has-background wp-block-paragraph\"><em>Visit Redwall Analytics to interact with the <code>{golem}<\/code>&nbsp;app: <a href=\"https:\/\/redwallanalytics.com\/2023\/02\/22\/nyed-data-explorer-shows-15-years-of-charter-school-success\/\">https:\/\/redwallanalytics.com\/2023\/02\/22\/nyed-data-explorer-shows-15-years-of-charter-school-success\/<\/a>.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the title image of this post, the app is filtered for the Subgroup and Test of \u201cAll Students\u201d and the \u201cELA\u201d, respectively. Using the ellipsis on the top left, try those filters, because the app initially shows all the subgroups and tests taken over the last 15 years, counting the same students in multiple groups. Once filtered, the population of public school students (Charter and DOE across all NYS counties) taking the test, should decline from 1.2 million to 900,000 over the period. When using the app, it is imperative to make sure that subgroups are selected which are mutually exclusive. For example, a \u201cWhite\u201d student may also be included in \u201cStudents with Disabilities\u201d, so choosing both of those subgroups would double count the total tested and possibly skew pass rates, while choosing S<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Again using our \u201cconditional drop-down filtering\u201d, the \u201cNeeds\/Resource Category\u201d is a NY State designation, which is helpful because it includes groups for NYC and for all Charters. My interest is largely Charter schools around NYC, so choosing \u201cNYC\u201d in the \u201cNeeds\/Resource Category\u201d offers only the five NYC Counties (\u201cQUEENS\u201d, \u201cBRONX\u201d, \u201cKINGS\u201d, \u201cNEW YORK\u201d and \u201cRICHMOND\u201d) in the County filter. After selecting those counties, changing to \u201cNeeds\/Resource Category\u201d to \u201cCharters\u201d filters for only Charters in those counties (essentially all NYC Charters in the Schools filter). It is then possible to choose individual schools in \u201cSchool\u201d tab. This field does allow to start typing a name which speeds up finding the school when there are many still in the subset. The results shown on the \u201cSummary\u201d tab aggregate all of the schools filtered with the tabs.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"494\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-1100x494.png\" alt=\"\" class=\"wp-image-185686 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-1100x494.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-700x315.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-300x135.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-768x345.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics-1536x690.png 1536w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_nyc_charters-redwall-analytics.png 2048w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/494;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: NYED Data<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">NYED Data Explorer filtered for All Students at NYC Charters<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As shown above, in the case of ELA, NYC Charters had a \u201cLift\u201d of 9.5% when filtered for \u201cAll Students\u201d in NYC (the \u201cLift\u201d of 18% for Math is not shown). \u201cLift\u201d is defined as the aggregated pass rate of the selected group, divided by that of all the other test takers at schools in those same geographies and subgroups (but not including the selected schools). The number of Charter students taking the ELA test has risen from ~6k in 2007 to ~66k in 2022. On the face of it, a 10% higher pass rate might not be so impressive, but the cohort at charter schools is only about 20% of all NYC students, so the proficiency of the overall system was significantly impacted by these schools. In addition, the demographics of the charter schools is also much different than the overall NYC population of students, and this matters, as we will show.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"505\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-1100x505.png\" alt=\"\" class=\"wp-image-185687 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-1100x505.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-700x322.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-300x138.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-768x353.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics-1536x706.png 1536w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_minority_charters-redwall-analytics.png 2048w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/505;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: NYED Data<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">NYED Data Explorer filtered for African American and Latino Students at NYC Charters<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We can switch our single subgroup from \u201cAll Students\u201d to two subgroups, such as \u201cBlack or African American\u201d and \u201cHispanic or Latino\u201d (shown above), because these are mutually exclusive. The \u201cLift\u201d in ELA rises to 40% for ELA, when the subset is compared to only to other NYC scholars in these subgroups, but not at charter schools. So attending charter schools has a much bigger effect for these two subgroups. We haven\u2019t shown, but leave it for the user to switch Test to \u201cMath\u201d to see the impressive 90% \u201cLift\u201d for Math for charter over DOE students). In the case of South Bronx Classical Charters, almost 90% of students fall into these two subgroup categories, representing a more appropriate basis for comparison. In the next step, I will filter to see the lift of the Classical Network against the most relevant cohort of those students in the NYC DOE system.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"508\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-1100x508.png\" alt=\"\" class=\"wp-image-185688 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-1100x508.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-700x323.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-300x139.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-768x355.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics-1536x710.png 1536w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_sbc_math-rewall-analytics.png 2048w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/508;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: NYED Data<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">NYED Data Explorer filtered for African American and Latino students at South Bronx Classical<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With these filters, we can see that almost all South Bronx Classical Charter students are tested, and just under 90% are \u201cproficient\u201d in Math, and there was almost no \u201clearning loss\u201d due to Covid, as there has been widely across most other schools in the United States. \u201cLift\u201d shows that students at South Bronx Classical are proficient in Math at almost 3x the rate as those in the DOE system. Pass rates for ELA are almost the same in absolute terms, but \u201cLift\u201d is lower than for Math at 138%, because the DOE pass rates are higher in that subject. Although Classical Charter Schools has been especially successful in navigating the Covid disruption, it is not the only network with persistent strong performance. Please also consider looking at Success, Icahn, Bronx Excellence and Zeta, which all have had excellent results over time, and there are surely others.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"502\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-1100x502.png\" alt=\"\" class=\"wp-image-185689 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-1100x502.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-700x319.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-300x137.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-768x350.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics-1536x701.png 1536w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/nyed_swb_math-rewall-analytics.png 2048w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/502;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: NYED Data<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">NYED Data Explorer filtered for Students with Disabilities at South Bronx Classical<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Although entry is by blind lottery, critics sometimes say that charters avoid difficult cases, but 106 Students with Disabilities (about 10% of all Classical test takers) were about 7x more likely to demonstrate proficiency in Math than at a district school. This is more a comment on the very small number of passing SWD at district schools, but is still striking to see what is possible. Students with Disabilities at Classical did better in absolute terms than the general population at most schools in the State. Another interesting subgroup, which is virtually gone from Classical in most years, are English Language Learners. Although 30% of entering students are ELL, with such high pass rates, most scholars pass the ELA in 3rd or 4th grade, and \u201ctest out\u201d of ELL status.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Future additions<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Making the data cleaning steps reproducible is one goal, but given the number of manual steps to collect the raw data out of MS Access databases, this is not easy. Now that the data is mostly clean, and the app is up and running, I will be adding further tables and graphics to aid in explorations to compare the trajectory of individual schools. Approximately 80 charters have come and gone for poor performance. It would be interesting to see how many failing DOE schools have also shut down, and the relationship between that and performance. We have narrowed our filters down to Counties, but zip codes might be even more revealing and not difficult to add. In his book \u201cCharter School and their Enemies\u201d, Thomas Sowell showed the striking differentials in performance of students in charter and DOE schools sharing the exact same buildings, so even this would not be too difficult to add.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The goal of the&nbsp;<code>NYED Data Dashboard<\/code>, as with many of my past \u201cRedwall Analytics\u201d projects, was to free valuable data from an inaccessible open repository, especially where the honesty surrounding the debate might be improved. My walk through is opinionated, but I think is supported by data, which is available for all to test and reproduce. Although I focused on NYC schools, the data includes all NYS public schools. The comments section of this blog is open, and I would welcome feedback on how to improve the app, different interpretations of the data or how to boost awareness of it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for Excellence do it?. <\/p>\n","protected":false},"author":1114,"featured_media":67374,"comment_status":"closed","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,338],"tags":[806,14817,8962,14819,14816,14813,14814,14815,14818],"contributors-categories":[14801],"class_list":["post-185643","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-ibkr-quant-news","tag-data-science","tag-golem","tag-math","tag-mdbtools","tag-nyed-assessment-data","tag-nyed-data-dashboard","tag-nyed-data-explorer-shows","tag-shiny-app","tag-shinywidgets","contributors-categories-redwall-analytics"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v28.5) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>NYED Data Explorer Shows 15 Years of Charter School Success<\/title>\n<meta name=\"description\" content=\"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/185643\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"NYED Data Explorer Shows 15 Years of Charter School Success | IBKR Campus US\" \/>\n<meta property=\"og:description\" content=\"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for Excellence do it?.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2023-02-28T18:00:06+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-13T16:45:53+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"900\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"David Lucey\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"David Lucey\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"David Lucey\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/992630f8a60ec8933943eef989b20a98\"\n\t            },\n\t            \"headline\": \"NYED Data Explorer Shows 15 Years of Charter School Success\",\n\t            \"datePublished\": \"2023-02-28T18:00:06+00:00\",\n\t            \"dateModified\": \"2023-03-13T16:45:53+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/\"\n\t            },\n\t            \"wordCount\": 2225,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/11\\\/binary-background-abstract.jpg\",\n\t            \"keywords\": [\n\t                \"Data Science\",\n\t                \"golem\",\n\t                \"Math\",\n\t                \"mdbtools\",\n\t                \"NYED assessment data\",\n\t                \"NYED Data Dashboard\",\n\t                \"NYED Data Explorer Shows\",\n\t                \"Shiny app\",\n\t                \"shinyWidgets\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Quant\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/\",\n\t            \"name\": \"NYED Data Explorer Shows 15 Years of Charter School Success | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/11\\\/binary-background-abstract.jpg\",\n\t            \"datePublished\": \"2023-02-28T18:00:06+00:00\",\n\t            \"dateModified\": \"2023-03-13T16:45:53+00:00\",\n\t            \"description\": \"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for Excellence do it?.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/nyed-data-explorer-shows-15-years-of-charter-school-success\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/11\\\/binary-background-abstract.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/11\\\/binary-background-abstract.jpg\",\n\t            \"width\": 900,\n\t            \"height\": 550,\n\t            \"caption\": \"Quant\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/992630f8a60ec8933943eef989b20a98\",\n\t            \"name\": \"David Lucey\",\n\t            \"description\": \"David Lucey, is the founder of Redwall Analytics. He spent 25 years working with institutional global equity research with several top investment banking firms. He completed his Masters of Science in Business Analytics at New York University in May of 2018. In its fifth year, this is selective program was built to train senior executives to find and communicate wisdom in data. https:\\\/\\\/redwallanalytics.com\\\/about\\\/\",\n\t            \"sameAs\": [\n\t                \"https:\\\/\\\/redwallanalytics.com\\\/about\\\/\"\n\t            ],\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/david-lucey\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"NYED Data Explorer Shows 15 Years of Charter School Success","description":"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/185643\/","og_locale":"en_US","og_type":"article","og_title":"NYED Data Explorer Shows 15 Years of Charter School Success | IBKR Campus US","og_description":"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for Excellence do it?.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/","og_site_name":"IBKR Campus US","article_published_time":"2023-02-28T18:00:06+00:00","article_modified_time":"2023-03-13T16:45:53+00:00","og_image":[{"width":900,"height":550,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","type":"image\/jpeg"}],"author":"David Lucey","twitter_card":"summary_large_image","twitter_misc":{"Written by":"David Lucey","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/"},"author":{"name":"David Lucey","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/992630f8a60ec8933943eef989b20a98"},"headline":"NYED Data Explorer Shows 15 Years of Charter School Success","datePublished":"2023-02-28T18:00:06+00:00","dateModified":"2023-03-13T16:45:53+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/"},"wordCount":2225,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","keywords":["Data Science","golem","Math","mdbtools","NYED assessment data","NYED Data Dashboard","NYED Data Explorer Shows","Shiny app","shinyWidgets"],"articleSection":["Data Science","Quant"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/","name":"NYED Data Explorer Shows 15 Years of Charter School Success | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","datePublished":"2023-02-28T18:00:06+00:00","dateModified":"2023-03-13T16:45:53+00:00","description":"Three years ago, in the course of building personal projects in R using public data from Connecticut, I wrote\u00a0How Does Stamford Charter School for Excellence do it?.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/nyed-data-explorer-shows-15-years-of-charter-school-success\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","width":900,"height":550,"caption":"Quant"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/992630f8a60ec8933943eef989b20a98","name":"David Lucey","description":"David Lucey, is the founder of Redwall Analytics. He spent 25 years working with institutional global equity research with several top investment banking firms. He completed his Masters of Science in Business Analytics at New York University in May of 2018. In its fifth year, this is selective program was built to train senior executives to find and communicate wisdom in data. https:\/\/redwallanalytics.com\/about\/","sameAs":["https:\/\/redwallanalytics.com\/about\/"],"url":"https:\/\/www.interactivebrokers.com\/campus\/author\/david-lucey\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/11\/binary-background-abstract.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/185643","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/1114"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=185643"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/185643\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/67374"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=185643"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=185643"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=185643"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=185643"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}