Aboriginal and Torres Strait Islander viewers are advised that this website contains the names and images of people who have passed
Aboriginal and Torres Strait Islander viewers are advised that this website contains the names and images of people who have passed
Methodology
Assessment Process & Methodology for The AJC RCIADIC Project
Project aims
It was an aim and requirement that this project be conducted in a way which is culturally responsive, safe, and Aboriginal-led; and that it support a self-determined approach to assessing implementation of recommendations based on Aboriginal Justice Caucus’ notions of progress and success.
To achieve this aim, the project was directed by a Project Steering Committee (called the AJC RCIADIC Working Group), which comprised of Aboriginal Justice Caucus members, including representatives from Aboriginal Community Controlled Organisations (ACCOs) and Chairpersons from Regional Aboriginal Justice Advisory Committees (RAJACs).
Problems identified in earlier assessments
Problems identified in earlier assessment and review processes:
- Too much focus on actions taken rather than on the impact or effect of those actions.
- Failure to assess whether action taken was relevant to the intent of a recommendation.
- Lack of analysis of the legitimacy and/or accuracy of agency self-assessments.
- Failure to consider Aboriginal Community experiences and views.
- No prioritisation of recommendations requiring further response.
How this project would overcome these deficiencies
This project aimed to overcome these noted deficiencies by:
- Gathering evidence on the impact and outcomes of action/s taken to implement recommendations.
- Analysing government/agency self-assessments in terms of the relevance of action taken to the intent of the recommendation, and the accuracy of claims.
- Prioritising recommendations requiring further action and response according to the degree to which they could reduce incarceration rates; increases safety in custody and advance Aboriginal self-determination.
Project principles and self-determination
The project took an Aboriginal-led approach which recognises the need for self-determined responses across all aspects of the justice system. The approach involved a Project Team of Victorian Aboriginal Legal Service (VALS) researchers responsible for delivering on the day-to-day requirements of the project, with the project governed and overseen at all stages by the AJC RCIADIC Working Group.
The Working Group contributed to project findings through a series of workshops responsible for review and sign-off of all project deliverables and key outputs, which enabled members to focus their efforts on the most impactful aspects of the project, including its design, and development and framing of key findings.
This approach is consistent with the first principle for ways of working set out in Burra Lotjpa Dunguludja (AJA 4), which is the prioritisation of self-determination and the transfer of power and decision making to Aboriginal groups.
Project phases
Phase 1: Establishment
Establish the foundations of the project, including through development of a high-level project plan and research protocol, and information gathering.
Phase 2: Quick Scan
Identify and prioritise thematic areas for more in-depth examination. Engagement with ‘justice system’ stakeholders was required to identify priority themes.
Phase 3: Thematic Deep Dive
Undertake thematic analyses of RCIADIC and coronial recommendations, implementation assessments, and progress updates from stakeholders responsible for implementation. This information was compared with qualitative and quantitative data (policies, manuals, evaluations, statistics, anecdotal reports, community experiences) that reflect the effectiveness of implementation.
Phase 4: Community conversations
Engage directly with Aboriginal people who have experienced various aspects of the criminal legal system to understand their needs and perspectives on what needs to change and improve in future. A flexible, trauma-informed and participant-centred approach was developed in collaboration with the Working Group.
Phase 5: Assessment and Reporting
In this final phase, the Project Team and Working Group came together through a series of co-production workshops to interpret the information, make assessments of implementation progress, identify and frame key findings and develop recommendations for reform. A final report was developed that drew together the findings and recommendations and was endorsed by the Aboriginal Justice Caucus prior to distribution.
The AJC RCIADIC Project Assessment Process
The AJC RCIADIC Project Assessment Process
Principles
Principles for developing assessment criteria included:
- Aboriginal values – criteria reflect Aboriginal values and perspectives on the most important aspects of implementation
- Consistency – criteria can be consistently applied by various AJC members, and can be consistently applied across recommendations (RCIADIC and coronial)
- Simplicity – criteria can be easily applied, easily understood and interpreted by various audiences (Aboriginal Community, agencies responsible for implementation etc)
- Replicable – criteria can be applied at different points in time to assess progress in future
Assessment Process
The AJC RCIADIC Working Group developed the assessment criteria and approach, which was endorsed by the AJC. The Project Team then provided detailed information for each recommendation which included:
- assessment status from previous reviews
- actions taken to implement the recommendation
- evidence of impact and or outcomes (we anticipate limited evidence of outcomes)
- the frequency with which the recommendation appears to have been breached (based on coronial findings, and concerns raised by Community members)
- related recommendations from other reviews, inquiries and Commissions.
Recommendations, grouped by theme, were presented to AJC members for assessment. AJC members provided an individual assessment and determined a consensus view of the implementation status for each recommendation.
Applying assessment criteria
there were four key questions to be answered to assess each recommendation:
- Does the action taken align with the intent of the recommendation?
- Is there clear evidence the desired change has happened?
- Is the recommendation still relevant today?
- Would full implementation reduce incarceration, increase safety in custody and/or progress Aboriginal self-determination?
These questions were assessed on several important criterion that reflect various aspect of implementation. A score of 0-3 was then applied (none, low, medium, high), with high scores representing a high priority.
Prioritising the recommendations
Given the vast number of recommendations examined in this project – 339 RCIADIC recommendations and approximately 70 coronial recommendations – it was important that where further action was required to achieve full implementation, those recommendations were prioritised so that efforts can be dedicated to those with the greatest potential to achieve significant improvements in justice outcomes for Aboriginal people.
Plotting the total score for the ‘action and outcomes’ criteria against the total score for the ‘relevance and potential impact’ assisted to prioritise recommendations for future work:
- Green zone: Moderate to high degree of action taken towards implementation with further action unlikely to have a significant impact = no need for further work.
- Yellow zone: Moderate to high degree of action taken towards implementation but further action likely to have additional impact on desired outcomes = some need for further work
- Red zone: Low to moderate degree of action taken towards implementation with further action necessary to have a significant impact = highest priority for further work
Recommendations that fall in the red zone are highly relevant with great potential to contribute to key Aboriginal justice outcomes but there is little evidence of action taken. These are the highest priority for further work to complete implementation.
The Project Team would transcribe the assessment and discussion of each Recommendation, while also providing a historical account for transparency and accountability.