Here, we provide an evaluation framework, to help you assess the social impacts of your neighbourhood battery project. This involves:
- A set of criteria, drawing on public values covered here,
- Some evaluation questions to flesh these out,
- Some indicators to help you see whether you are delivering on values, and
- Some methods and sources to help you assess and document against the criteria.
All of these criteria will likely be important to people for most neighbourhood battery projects, so we recommend giving some attention to each of them. However, some may be more of a priority to your community or project. It is important that your evaluation is guided by your purpose and objectives and this will help you prioritise and allocate time and resources. There may also be additional values that are important to your project that are not listed here. By all means add these to your evaluation.
For large projects, we recommend commissioning an independent party to conduct the evaluation. They can provide both expertise and independence, which will strengthen the evaluation.
The framework is focused on positive impacts (reflected in the indicators). Negative impacts are often simply the absence or reverse of positive impacts (the battery is not efficient, or there is limited transparency, or wellbeing is reduced). However, be aware that there may be additional negative impacts that are unforeseen. For example, the battery may create controversy and division within the community, or there may be a particular group that has unintentionally been excluded. It is tempting to paint a picture of success that frames out any negative impacts, but they are likely to be the things that cause conflict and problems down the track. Bringing a critical perspective to your evaluation can help you to pick up problems in time to mitigate them.
| Overall evaluation question | To what extent is this neighbourhood battery project contributing positive social impacts, avoiding or mitigating negative impacts, and meeting community values and expectations? |
Public expectations
| What is being evaluated? | Evaluation questions | Indicators | Methods & Info sources |
|---|---|---|---|
| Effectiveness & efficiency | Has this project: successfully delivered a community battery? Made good use of available time and resources? | Battery is up and running. Project is on time and budget | Provide evidence of an operating battery; Project reports Project team, commentators |
| Trustworthiness | Is this project being planned/conducted by a trusted organisation? Are there processes to increase trust (transparency, accountability)? | Community members trust the proponent Processes are transparent and accountable to community | Surveys, feedback forms Project reports Community, Project team |
| Fairness | Have decisions been made fairly? Does the project distribute profits and benefits fairly across the community? | Community members judge decisions, processes and benefit sharing to be fair | Surveys, feedback forms Community, NGOs |
| Inclusiveness | Has the project enabled participation of diverse community members? | Participants reflect the diversity of the community | Gather stats on diversity Community, social NGOs |
| Customer experience & protection | Have experiences of participation been positive? Can participants easily access relevant information? Is there a process for dispute resolution? | Participants are satisfied. Processes are in place | Surveys, feedback forms Project docs Customers, participants, installers, staff |
Sound governance
| What is being evaluated? | Evaluation questions | Indicators | Methods & Info sources |
|---|---|---|---|
| Sound project governance & accountability | Are there clear mission, goals, values and processes for decision making and dispute resolution? Is there transparency & accountability? | Project governance processes exist and are implemented | Project docs Testimonies Website, project team, community |
| Ethical data governance | Are there security and privacy provisions in the use of data? | Provisions exist and are used. | Project docs, Testimonies Project team, partners |
| Evaluation & learning | Has the project been evaluated for accountability, improvement and learning? Have lessons been shared? | Evaluation has been conducted and results are publicly available Lessons have been shared. | Project docs Evidence of sharing Project team, wider NB community |
Positive community impacts
| What is being evaluated? | Evaluation questions | Indicators | Methods & Info sources |
|---|---|---|---|
| Financial and practical impacts | Has the project left community members better off financially, without giving them unacceptable burdens? | Costs associated with energy use by community members are reduced. Community members find requirements on them acceptable and not burdensome. | Energy bills, surveys, feedback forms Community |
| Equity | Has the project reduced inequity in the community, especially with respect to energy? Has it reduced energy poverty? | Costs associated with energy use by poorer community members have been reduced more than for richer ones. Non-participants are not disadvantaged. | Energy bills, testimonies, surveys Community members, social NGOs |
| Wellbeing & empowerment | Has the project contributed to community wellbeing? Has the project empowered community members? | Community members feel better as a result of the project. They feel more empowered. | Surveys, feedback forms Social media discussion Community |
| Economic & community development | Has the project contributed to economic opportunities and community development? | The project has led to economic opportunities (beyond energy bills). Community assets and/or capacity have increased. | Evidence of jobs, business esp local Testimonies, observation Community members, local & national businesses |
Better energy system
| What is being evaluated? | Evaluation questions | Indicators | Methods & Info sources |
|---|---|---|---|
| Climate action and energy transition | Does the project contribute to decarbonisation and support renewables? Does it connect with other climate action? | The project has made a positive contribution to reducing emissions. It has supported climate action. | Decarbonisation & hosting capacity calculations Testimonies, surveys Community |
| Energy practices | Has the project led to changes in energy practices (e.g. load shifting, demand management, reduced consumption, energy efficiency measures)? | The project has stimulated changed practices. | Testimonies, surveys Social media discussion Community |
| Security, stability & resilience | Has the project contributed to an energy system better able to integrate Community Energy Resources without problems for the network? Does it give people trust and confidence in the energy system? | The project has delivered without detriment to the stability of the network. Community members have greater trust and confidence in the energy system. | Testimony from DNSP; Testimony, surveys from community members |
| Safety & lifecycle impacts | Has the project put in place plans and strategies to minimise the environmental and social impacts of the manufacture, installation, running and recycling/disposal of the battery? | Plans and policies exist and are enacted to minimise impacts in purchase, installation, running and end-of-life. | Plans & policies, evidence of implementation Project team, partners |