-
Notifications
You must be signed in to change notification settings - Fork 23
TeamPSD Policy & Procedure SOP
- Policy
- Workgroups
- Definitions
- GitHub Labels Table
- Kanban Management System
- pull_requests
- Monthly Sprint Management
- Workgroup Leads & Support Workgroups Meeting
- Appendices
This policy applies to all TeamPSD members involved with research under grants, R21DA042198, R01DA046651, IIRI01HX002521 commonly known as Modeling to Learn (MTL). This policy is subordinate to new or existing Veterans Administration (VA) policy. Any issues identified that are contrary to VA policy or federal laws should be brought to the attention of the Principal Investigator, Lindsey Zimmerman, ([email protected]).
This policy details the governing principles, definitions, responsibilities and procedures for managing cards, issues, & pull requests in the Kanban production management system for the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker GitHub projects for the TeamPSD. Finally, the policy will describe how MTL and overarching research experimental design and reporting with be coordinated.
Most of our work is public-facing with the exception of any items that include Protected Health Information (PHI) or Personally Identifiable Information (PII). All of our public materials are free to download and use. We want our insights and resources to benefit the larger community. We use R, a free open-source coding language and a specific file naming convention to ensure all of our materials are machine & human readable (all lower case, no spaces, with dates as yyyy_mm_dd i.e. teampsd_guiding_principles_2020_01_01). Use email for any private discussions. Host any private files in password-protected locations or in folders behind the VA firewall. When in doubt, ask an HQ member or err on the side of caution. Make sure your work and accompanying documentation allows other team members or scientists in the field to reproduce and understand your work without further questions.
Keep in mind we work under the federal government of the United States for the Veterans Health Administration (VHA), the largest integrated healthcare system in the country. Everything we produce is a reflection of the VHA. We work with a wide range of nationally-distributed partners both internal and external to VA, including very important and high-level stakeholders. Double check the role and responsibility of people you are communicating with before you do. With these partners, we work in a participatory learning manner and iterate based on feedback from the field to ensure our work is responsive to ongoing changes.
Ask questions early and often to prevent escalating issues down the road. Refer to existing resources (cheatsheets, checklists, etc.) as well for clarification. Double check all work before handing it off to the next team member to reduce rework. Think through dependencies across the team and partners and prioritize work based on the most recent information you have. Manage workflow asynchronously (via GitHub) and only schedule meetings when absolutely necessary.
4. Use effective communication (across all types of communication including emails, GitHub, Lucid, etc.).
Assume everyone you’re communicating with is smarter than you and cares more than you and is busier than you. Use clear and concise language, with formatting (bold, underline, bullets, etc) to emphasize the main decisions or issues making sure to include the "Who" "What" and "When". Always include the full context and details necessary to make an informed decision (and make sure you are up to speed on the context and details before responding). Use complete sentences as much as possible and write in the active voice for clarity. Often the team looks back to previous records of meetings, GitHub discussions, etc. to remember and track decisions that were already made or if they missed the meetings where an issue may have been discussed. As such, we always need to keep the most accurate record possible to reduce rework and provide clarity. Use emojis or humor (as appropriate) to help maintain a positive and collegial vibe. Turn on your webcam as much as possible for the face-to-face interaction.
We work daily with team members and partners that are experts in their respective fields, and it is easy to lose track of a complex discussion. We've found that reflecting-back requests and decisions in your own words has been the best method to reduce miscommunication and to keep track of all of our decisions or issues accurately. If you ever need help while scribing, always "tap" someone else on the team for assistance. Ask for clarification and slow down if necessary.
-
Principal Investigator - Provides overall direction and guidance to Workgroup Leads. Coordinates research activities and prioritizes activities within the MTL program with the HQ workgroup
-
Co-facilitators - Gathers field information and provides feedback to Workgroup Leads regarding the performance of a product in the teaching/learning environment. Facilitate Modeling to Learn 12 Session Program with identified clinics for the R01 and IIR grants.
-
Co-investigators - Oversees the science and research dependencies across the project.
-
Workgroup Leads - Oversees the production and maintenance of their products in terms of quality, cost and delivery, responding to the needs of the projects as defined by co-investigators and co-facilitators
- Checks the bug_tracker, feature_tracker, document_tracker, and manuscript_tracker (as related to the workgroups) daily to identify & assign interdependencies from the labels list used as a dependency check
- Ensures that all cards are updated in the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker (as relevant to the workgroup) in include a due date once they have been triaged with HQ & other workgroup leads.
- Trains respective workgroups in changes to policy & procedures for Kanban workflow.
- Estimates the time required to fulfill completion of bugs (issues) & features.
- Attends weekly Monday 8am PST / 11am EST Workgroup Leads meeting or alerts the HQ Proxy (see below) in their stead, should they not be able to attend.
-
Proxy - An individual who is a member of HQ that supports a specific workgroup and can participate in the absence of the Workgroup Lead to represent the interests of a workgroup. A Proxy has the decision-making authority of the Workgroup Lead they represent. The Workgroup Lead must still provide detailed and concise documentation of questions and scope on bugs and features related to their workgroup in the Workgroup Leads meeting agenda.
The table below describes all of the TeamPSD workgroups including their Workgroup Lead, Meeting Time, and Role. For each meeting, it is the responsibility of the Workgroup Lead or HQ point person to:
- Set the agenda
- Check RSVP’s for attendees
- Clean up meeting notes for clarity
- Send the follow-up email
- Publish the meeting to Basecamp
- Save Lucid audio in the TeamPSD folder: Meeting Agendas & Recordings (and backup and save any mtl.how/live recordings as relevant)
| Workgroup (Workgroup Lead) (Meeting Time) | Role |
|---|---|
| Facilitate/EES (Jane) (Mon 10:00-11:00a Pacific, Wed 9:30-11:30a Pacific, Fri 9:30-11:30a Pacific; 3rd Friday 9:00-11:00a Pacific) | Provides MTL program resources such as learner SEE and facilitator SAY scripts, checklists, guides, EES (Employee Education Services) brochures, and post-tests; and co-facilitates the MTL program. *Rita is the HQ point person for this workgroup. |
| Grants (Lindsey) (Tues 10:00a-11:00a Pacific) | Develops and maintains aims, strategies, and research plan for grants |
| Headquarters (Lindsey) (Daily Check-In 12:30-1:00pm Pacific,Thurs 1:00-2:00pm Pacific,4th Friday 12:30-4:00pm Pacific) | Manages oversight of all workgroups, identifying interdependencies and parallel workstreams. Provides guidance on prioritization. |
| Manuscripts, Publications, and Conferences (Lindsey) (Mon 9:00a-12:00p Pacific)(Tues 10:00a-11:00a Pacific) | Develops and maintains manuscripts, publication schedule, authorship agreement, and conference materials |
| Modeling (James & Tom) | Builds models of systems that support clinician experimentation. |
| Qualitative (David) (Tues 8:00a-9:00a Pacific) | Codes team qhfd inputs for qualitative analysis and development of future fidelity materials. *Jessilyn is the HQ point person for this workgroup. |
| Quantitative (Anthony) (Thurs 10:00a-11:00a Pacific) | Provides data and analysis of data that supports other workgroups and research. |
| Simulation User Interface (James) | Provides an accessible, web-based user interface for practitioners to experiment using simulation. |
| Workgroup Leads & Support Workgroups (Stacey) (Mon 8:00a-9:00a Pacific, Thursday 8:00a-9:00a Pacific) | Triage all issues that have entered into the issue-tracker triage section and identify workgroup milestones and action items for the month. Rearrange deadlines and interdependent timelines in response to emerging issues. |
| VAPOR (Jennifer) (TBD) | Provides veteran perspective and guidance on Modeling to Learn initiatives |
- Bug - An existing feature that has been developed and is not working correctly.
- Issue - An important topic or problem for discussion. Issues are documented in the GitHub Issues tab by selecting “New Issue.”
- Deliverable - A product of a task or group of tasks.
- Expedite - To move the priority of a bug or feature to the top of a queue. Expediting is poison to a production system and should be used only in exigent circumstances.
- Feature - A characteristic, attribute or topic that requires work breakdown, research, design, development and testing. Features can be tagged as “fast_track” in order to expedite its design and development. Features are documented in the GitHub Issues tab by selecting “New Feature”
- Kanban - A board that contains issue cards and is used to communicate the status and priority of bugs, features, documentation, & manuscripts as they move through the production process. There are four Kanban to manage production in the MTL program: bug_tracker, feature_tracker, document_tracker, and manuscript_tracker.
- Task - A cognitive or kinetic behavior that consumes time. A task or group of tasks are necessary to create an outcome or deliverable.
- Labels - A tag in GitHub that is affixed by an originator or workgroup as a means to identify the task holders for a specific issue. Workgroup leads can filter by labels in the tracker to sort for their specific workgroup or other workgroups’ Kanban cards in the search bar found in the upper right-hand corner of each tracker (below). To use the filter function, use the code “label:labelname” i.e. “label:facilitate” or “label:sim_ui”. (You can use this function with other sort options as well i.e. author:lzim, etc.). In most cases, an issue should only have 1 label at any given time.
Labels shall be assigned a color, expressed in lowercase and use an underscore in lieu of a space. Below is the list of labels and their purpose:
| Label | Purpose |
|---|---|
| document | expresses a dependency for at least 1 of 5 levels (describe, detail, document, disseminate, depend) of documentation to be tracked on the document_tracker. The point person for each level of documentation will be responsible for checking the issue & feature tracker for dependencies (describe: Jane & Debbie; detail: Tom & Lindsey; document: Stacey & HQ; disseminate: Lindsey, Anthony; depend: Lindsey, Jane, & Jessilyn) |
| facilitate | alert all of the facilitate workgroup to an issue that may affect facilitation (Lindsey, Jane, Debbie, Tom, David, Claire, Gayle, John, Matt, Jay, Theresa, Marcia) |
| hq | alert to HQ workgroup (Lindsey, Stacey, Rita, Jessilyn, Jennifer) to track |
| manuscript | expresses a dependency on manuscripts to be tracked on the manuscript_tracker |
| model | alert to Tom & James of a dependency on the model workgroup |
| pi | alert to PI/Lindsey that an action or decision is necessary |
| qual | alert to David, Kathryn, Swap, Lindsey, Jessilyn, & Stacey to of dependency on the qual workgroup |
| quant | alert to Anthony & Ash of dependency on the quant workgroup |
| sim_ui | alert to sim_ui workgroup lead of an issue or dependency that affects the sim_ui. The workgroup lead will evaluate sim_ui impacts and collaborate with other workgroup leads to determine an adequate resolution. |
| urgent | indicates a short amount of time is available for resolution and needs to be prioritized by workgroup |
| ees | description to come |
| vapor | description to come |
The issue_tracker is divided into 6 columns described below. The purpose of the issue_tracker is to provide triage and track the disposition of issues that require action by one or more workgroups. Issues labeled as “bugs” will be tracked here.
-
needs_triage - This column is the main intake for all issues assigned to the issue_tracker. All issues requiring a disposition will land here. When an issue lands in this section, any team lead may review it and alert other leads as to the action required.
-
validated_actions (ranked) - This column contains all issues that have received a “bug” disposition by a Workgroup Lead(s) and has been provided sufficient details to fix the problem. This is a rank-ordered list based on due dates indicated in the title. The rank-order of this section can be changed at any time. If an issue is determined to be a “feature,” it will be placed in the feature_kanban (see feature_kanban below).
-
under_development - Bugs that are currently under development are listed in this column. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workgroups.
-
testing - Bugs that are currently being tested are listed in this column. Some issues may skip this step and go directly from under_development to done.
-
done - This column contains completed issues. Responsible Workgroup Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
-
closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.
The feature_tracker is divided into 9 columns described below. The purpose of the feature_tracker is to report and maintain information regarding the analysis of dependencies and work-content, and track the progress of issues that require development. Issues labeled as “features” will be tracked here.
- work_breakdown - Validated feature requirements that have received a disposition are listed here. Issues in this column are analyzed by Workgroup Leads for dependencies, effort content and milestones they may support (see Appendix 1 - issue_template). Issues will progress from this section to either the operations_to_do (ranked) or the research_to_do (ranked) sections.
- operations_to_do (ranked) - Features that require research, design and development of products in one or more workgroups are listed here in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
- research_to_do (ranked) - Features that support research, evaluation and documentation efforts directly related to supporting grants or higher-headquarters reporting requirements are listed there in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
- under_development - Operations or research features under development are listed here. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workgroups.
- functional_testing - Features that are currently being tested for basic functionality (i.e. does it work?) are listed here. Some issues may skip this step and go directly from “under_development” to “done.”
- done - This column contains completed issues. Responsible Workgroup Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
- closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.
- future_release – This column contains unresolved feature ideas that would be great to include in a future MTL release, but currently is not a pressing need.
The document_tracker is divided into 5 columns described below. The purpose of the document_tracker is to track documentation dependencies at 5 levels for each of the 12 sessions of Modeling to Learn. There will be a card for each session of the 12 sessions in each of the 5 Kanban columns which will be closed & reopened as interdependencies get identified.
Each column also has a “meta” card which is used to indicate interdependencies that are relevant to all/most of the 12 sessions as well as policy & workflow decisions specific to that documentation column.
- describe_learners – Documentation dependencies relevant to learners, including SEE guides
- detail_facilitators - Documentation dependencies relevant to facilitators, including SAY guides & one-pagers
- document_team - Documentation dependencies relevant to TeamPSD, including cheatsheets
- dissemintate_scientists_va - Documentation dependencies relevant to co-investigators & larger scientific audience, including progress reports, code, grants, etc.
- depend_products - Documentation dependencies relevant to other MTL products, including videos
The manuscript_tracker is divided into 10 columns described below. The purpose of the manuscript_tracker is to track progress of manuscripts through 10 major stages until ready for publication.
DO NOT post any direct manuscript content to GitHub; all drafts and related materials will be posted on the relevant OSF project. There will be a card per manuscript that moves through the tracker.
- 01_osf_project – OSF project is created for this manuscript with all relevant materials posted, Zotero & GitHub integrations approved, cheatsheets linked, and relevant people added
- 02_authorship – Initial authorship weights and task division are identified
- 03_write_analyze – Manuscript is written and materials for analysis are produced.
- 04_edit – Manuscript goes under rounds of editing and revision between writing team.
- 05_approve_letter – Team working on manuscript gives final approval and drafts letter to editor
- 06_submit_under_review – Manuscript is submitted to relevant journals and is undergoing review
- 07_revise_and_respond – Manuscript feedback is received from journals and team working on manuscript revises it accordingly
- 08_resubmit – Manuscript is resubmitted to relevant journals
- 09_accept – Manuscript is accepted for publishing by journals
- 10_publish_publicize – Manuscript is published!
Updates to MTL documentation are tracked in the document_tracker. The document_tracker represents the entire corpus of Modeling to Learn documentation at any given time. The document_tracker state for any MTL release is that all cards in the tracker are "closed." This means that that all documentation is up-to-date, was reviewed and moved to production consistent with the current release of Modeling to Learn. Each card in the document_tracker includes a checklist that must be completed before the document is ready for release.
Pull requests that are completed based on a document_tracker checklist should reference both the document_tracker card, the GitHub task issue assigned to them using cross-referencing with the relevant #/hashtag number.
LZ Edits are still needed below here Feb. 20, 2020 Reviewers – Workgroup leads will develop code review procedures in their own workgroups. Including who will regularly review and merge pull requests each morning. Reviewers provide the final approval on commits, and merges and closes pull requests(merge & delete branch). Assignees – Execute the task by creating a pull requests, make sure pull request is linked to appropriate card in the document_tracker, and the issue task that describes
- The main purpose of developing a monthly sprint milestone timeline is to help everyone in each workgroup look into the current and following month’s timeline and organize all possible dependencies and delegations. This will meet the need of how people can see where they are needed, reduce bottlenecking, and accomplish deliverables.
- The GitHub Milestone feature will be used to track high priority goals that must be accomplished by the end of the month. Issue, feature, manuscript, and document cards will be attached to a Milestone(s). Click here on how to create milestones.
- Milestones must be identified and outlined alongside their time cost (how long will it take to complete this and when is the earliest that it can be completed) by each workgroup. It is the responsibility of the Workgroup Lead to bring the draft Milestones to the 1st Workgroup Leads meeting of each month.
- The PI will join the 1st Workgroup Leads and Support Workgroups meeting of each month to provide clarification and guidance on that month’s Milestones. The 4th Tuesday All Hands meeting will be used to celebrate the successes and completion of each workgroup's monthly milestones.
When? Workgroup Leads meetings occur every Monday from 0800 PST to 0900 PST and Support Workgroups meetings occur every Thursday from 0800 PST to 0900 PST. If an ad-hoc Workgroup Lead and Support Workgroups meeting is required, any lead, Co-I or the Principal Investigator may coordinate a meeting with all other leads to resolve urgencies.
Where? Meetings will be hosted on-line using Lucid. Regular Workgroup Lead meetings are scheduled and managed by HQ. Screen-sharing will be facilitated by Adobe Connect at mtl.how/live.
Goal The goal of the meeting is to triage all issues that have entered into the issue-tracker triage section and identify workgroup interdependencies based on priorities for the week. Deadlines will be added or updated at this meeting based on changing priorities and scope.
The PI will join the 1st Workgroup Leads and Support Workgroups meeting of each month. HQ will provide a draft dev/test/prod schedule outlining priorities, deliverables, and review dates across all workgroups for the month.
Governing Principles
- Prepare - if something is to be presented, be sure it is loaded up to Lucid meeting 24 hours prior to meeting and added to the New Business area of the agenda.
- Document the discussion - take notes!
- Keep the meeting moving along - stay on topic.
- Stay aware of the time - a good meeting starts on time and ends on time.
Standard Agenda
- Role call
- Assign note-taker (this should be a revolving duty). Note-taker will track decisions in Lucid and next steps or questions related to issues will be tracked directly in the issue thread.
- Review and Triage open issues (on the mtl.how/issues tracker)
- Review Old business - Items that were New Business in prior meetings where an update is necessary to update or close the issue.
- Discuss New business - an item that affects the progress of an issue currently under development that may require discussion and decision. If a lead has an item of new business, it should be entered into the agenda prior to the meeting time. It should answer the following at a minimum:
- What is it?
- What does it affect in your workflow?
- What do you need to resolve the issue?
- When do you need it?
Title: Deadline (to be added or edited during Workgroup Leads or Support Workgroups meeting) + Product: Short description (ex. 8/12 AGG: Missing feedback loop)
1. Add description:
<< Paste screenshots here>>
2. Click on the right:
a. Projects - assign to issue_tracker.
b. Assignee - assign the team member producing work for the issue.
c. Labels- review and select a label to indicate the responsible workgroup
d. Milestones - select the dependent milestone.
Workgroups Leads Identify Constraints (check all that apply):
- Capacity
- Cost
- Schedule
- Interference
- Capability
- Other (describe)
Add estimated person-hours to complete:
Edit the due date (if necessary)