IN THIS ARTICLE
Summary
Health Check
Topics
By Member
Health Check Data Dictionary
Each Yellowdig Community contains a robust data analytics dashboard for measuring and improving Community Health, which is viewable by Community Owners, Community Facilitators, and Organization Administrators. Yellowdig measures Community Health along three dimensions: Sharing, Listening, and Interacting. Each dimension of Community Health is reduced to a composite percentile score, and the "ingredients" of each dimension can be visually compared to those of the average Community in Yellowdig or in one's immediate Organization. The analytics dashboard also includes an automatic best practices checker, a breakdown of Topic usage per Community Member, and a sortable Member-level activity table.
The Community Health dashboard was designed to be an evidence-based pedagogical tool. You can use the dashboard to visualize usage and performance patterns, compare Organization-level success metrics to Yellowdig-wide metrics, identify highly successful instructors and departments, and compare students' levels of productivity, connectivity, and attentiveness in Communities with different point settings and content strategies. The dashboard helps instructors and administrators infer correlations between recommended practices and success outcomes, identify areas of excellence and improvement in their Communities, and further hone their Yellowdig pedagogies.
Summary
The Summary tab of the Community Health dashboard gives Community Owners and Facilitators snapshots of their Community's overall and week-by-week engagement levels. While the Health Check tab provides more detailed metrics, the Summary tab focuses on just four key metrics: Total Posts, Total Comments, Total Reactions, and the Conversation Ratio. Of these four metrics, the Conversation Ratio is most important. It reflects the extent to which students are paying attention to each others' work and engaging in back-and-forth conversation. (We recommend aiming for a Conversation Ratio of at least 8.0).
By default, the Summary page shows statistics for the entire life of the Community. To see activity for just one week, select the drop-down list under Showing Activity For and choose the desired week. To learn more about each of the four key metrics, click the tooltip in the lower-right hand corner of each card (just under the drop-down list). To see which weeks were most productive in terms of Posts, Comments, Reactions, or the Conversation Ratio, scroll down to the Activity Log section of the page and select the corresponding column name to sort by it.
Please click here if you're unable to watch the video below.
Health Check
The Health Check is the heart of the Community Health dashboard. The Best Practices Checklist helps instructors verify that their Community settings align with our evidence-based passive best practices and facilitates remediation where necessary. The Community Health Composite Scores give instructors a more fine-grained picture of their Community's strengths and weaknesses. Our three core composite scores are Sharing, Listening, and Interacting, which are derived from measures of how much students share with their peers, listen to their peers, and interact with their peers, respectively.
Sharing, Listening, and Interacting scores range from 0 to 100.* For most curricular Communities, these interpretive rules of thumb apply:
- 0 to 24: Deficient
- 25 to 74: Improvable
- 75 to 100: Healthy
It is not possible and usually not advisable** to try to achieve "perfect scores" of 100, but generally instructors should strive for scores of at least 75 across all three areas relative to the Yellowdig Global Medians. Achieving a good balance is probably advisable, but based on our data it's far more important to have high Listening and Interacting Scores than it is to have high Sharing scores. High Listening and Interacting Scores reflect that students are consuming the content that is being posted and are actually interacting about it. Students do not specifically receive points for any of the behaviors in the Listening and Interacting score areas so when they are high they also tend to represent "voluntary" participation. When these are high Sharing is often brought along with them because students are starting to post and comment voluntarily (i.e., more than required). A high Sharing score in the absence of Listening and Interacting usually happens when students are required to post a lot or write long posts, which does not usually lend itself to overall healthy community formation. Typically this will lead to specific variables that have exceedingly high percentile scores while others are very low.
By default, your Community Health Composite Scores are calculated based on Yellowdig's global distribution of outcomes. If your Organization contains enough Communities, you will have the option to select your own Organization for comparison. The selectable Sharing, Listening, and Interacting tabs feature graphs that show how your Community compares to Yellowdig Global or to your selected Organization with respect to component metrics.
To follow a best practice that your Community is not yet following, click on the best practice in question. You will either be taken to the page where you can change the setting in question or receive remediation instructions. To change your comparison Organization (if available), click on the field next to Comparing with:. To see how your Community compares to your chosen Organization along specific metrics, click on the Sharing Score, Listening Score, and Interacting Score tabs. For explanations of what each metric means and how it's calculated, click on the tooltip in the lower-right hand corner of each visualization.
Please click here if you're unable to watch the video below.
*Each composite score for sharing, listening, and interacting is a simple average of the percentile scores of each of the constituent metrics that make up the composite score. These variables are the variables that appear below the score when you click on the individual scores in the dashboard, as seen in the video above. The reference sample for health scores can be toggled from Global (all Yellowdig communities) to only communities in your own organization's network, once there are enough communities in your organization. This toggle will usually change where the current community ranks relative to other communities and updates the percentile scores. Depending on whether communities at your institution are relatively healthy compared to the average YD organization, your scores may appear lower or higher when you toggle to between Global and your network.
**It is not realistic to expect to get all scores and their underlying variables to 100 because many variables will naturally fall when others are maximized. For example, if you try to require students to create more and more posts to get to the 100th percentile on posts, they will tend to create fewer comments. If you make students write really long posts, at some point they will start to view fewer posts because they are spending all their time writing and the posts are giant. Nonetheless, all of the variables do tend to go up as learners are participating more. Learners obviously tend to voluntarily participate more when they are seeing value in participating in your community, which is why these composites are informative. A community health score total above 200 (for all 3 areas added) is a relatively healthy total score and a good goal for a standard course-based community. The absolute highest "standard" communities tend to achieve around 250. Most communities above that are communities for other aspects of learner engagement (e.g., orientation, clubs) or specifically have some aspect of their setup which is unusual (e.g., highly intensive short-courses or really high point requirement, etc.).
Topics
The Topics tab gives instructors a bird's eye view of what their students are talking about. Overall Topics Activity provides a high-level breakdown of topic-related conversations by number and percentage of posts. Topic Activity per User provides a fine-grained breakdown of each student's posting activity by topic.
To filter Overall Topics Activity by a specific topic, enter the name of the desired topic in the Filter Topics field. To sort topics by number or percentage of posts, select the respective column names. To filter Topic Activity per User by a specific user, enter the name of the user in the Filter Users field. To sort users by the percentage of posts dedicated to a specific topic, select the column name that matches the desired topic. To see a selected user's posts, select the name of the user.
Please click here if you're unable to watch the video below.
By Member
This tab contains an abbreviated summary of each Community Member's activity, including point totals. (For more detailed member-level reports, go to Reports → Points Report or Reports → Member Report.) To sort Members by activity or points, select the corresponding column name. Click on the table and use the arrow keys to scroll left or right in the table. To see a Community Member's point log, select the name of the Member.
Please click here if you're unable to watch the video below.
Health Check Data Dictionary
health_dict = {
'Avg # of posts': 'The total number of posts in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'Avg # of comments': 'The total number of comments in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'Avg # of links shared': 'The total number of hyperlinks shared in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'Avg total word count': 'The total number of words written in posts or comments in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'Avg # of multimedia items shared': 'The total number of attachments or canvas drawings shared in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'Conversation ratio': 'The total number of comments in the course divided by the total number of posts in the course',
'Avg # of hyperlinks clicked': 'The total number of times hyperlinks shared in the course have been clicked, divided by the number of members in the course, divided by the number of days since the course start date',
'Avg # of posts viewed': 'The total number of times posts have been viewed, divided by the number of members in the course, divided by the number of days since the course start date',
'% of posts with 1 or more comments': 'We consider a post to be orphaned if it never received any comments. This metric is calculated as 1 - (number_of_orphaned_posts_in_the_course / total_number_of_posts_in_the_course)',
'Avg # of reactions given': 'The total number of reactions added to posts in the course, divided by the number of members in the course, divided by the number of days since the course start date',
'% of connected users': 'The total number of members who have either been mentioned in a post or received a comment from another member on their post, divided by the total number of members in the course',
'Avg # of connections': 'The total number of (comments, reactions and mentions) divided by the number of members in the course'
}Audience: This help article is for Instructors, Designers, and Administrators. Students or Learners cannot access these settings.