Skip to main content
  • New Internet Architecture Board, IETF Trust, IETF LLC and Internet Engineering Task Force Leadership Announced

    Members of the incoming Internet Architecture Board (IAB), the IETF Trust, the IETF Administration LLC (IETF LLC) Board of Directors, and the Internet Engineering Steering Group (IESG)—which provides leadership for the Internet Engineering Task Force (IETF)—have been officially announced, with new members selected by the 2021-2023 IETF Nominating Committee.

      13 Feb 2023
    • Informing the community on third-party correspondence regarding the W3C

      In accordance with our policy of transparency, this blog post is being published in order to keep the community informed about recent correspondence with lawyers acting on behalf of the Movement for an Open Web.

      • Lars EggertIETF Chair
      8 Feb 2023
    • Six Applied Networking Research Prizes Awarded for 2023

      Six network researchers have received Internet Research Task Force Applied Networking Research Prize (ANRP), an award focused on recent results in applied networking research and on interesting new research of potential relevance to the Internet standards community.

      • Grant GrossIETF Blog Reporter
      9 Jan 2023
    • Travel grants allow Ph.D. students to participate at IETF meeting in-person

      Sergio Aguilar Romero and Martine Sophie Lenders, both Ph.D. students in technology fields, attended and participated in the IETF 115 meeting in London with assistance through travel grants from the Internet Research Task Force.

      • Grant GrossIETF Blog Reporter
      7 Jan 2023
    • Impressions from the Internet Architecture Board E-Impact Workshop

      The IAB ran an online workshop in December 2022 to begin to explore and understand the environmental impacts of the Internet. The discussion was active, and it will take time to summarise and produce the workshop report – but the topic is important, so we wanted to share some early impressions of the outcomes.

      • Colin PerkinsIAB Member
      • Jari ArkkoIAB Member
      6 Jan 2023

    Filter by topic and date

    Filter by topic and date

    IETF 111 post-meeting survey

    • Jay DaleyIETF Executive Director

    23 Aug 2021

    The results from our IETF 111 post-meeting survey are now available.

    The survey results for the IETF 111 post-meeting survey are now available on a web-based interactive dashboard.  We are always very grateful for the detailed feedback that we have received and will continue to process over the next few months.  The commentary below highlights a couple of areas where changes we have made based on feedback have been a success, and areas we still need to work on.  

    Analysis

    In total 166 responses were received.  Of those 164 participated in IETF 111 from a population of 1329 giving a margin of error of +/- 7.17%. The total numbers of meeting participants has grown (1196 for IETF 110, 1282 for IETF 109) while the survey response rate is significantly down on previous meetings (299 for IETF 110, 258 for IETF 109).

    The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1.  While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages.

    In this commentary a comparison is made with the IETF 110 Meeting Survey results using a comparison of means that assumes the two samples are independent even though they’re not but neither are they fully dependent.  A dependent means calculation may give a different result. A few comparisons are also made using a comparison of proportions.   

    Overall satisfaction

    The mean satisfaction score for IETF 111 (Q10) was 4.13 with 87% either ‘Satisfied’ or ‘Very satisfied’.  This is a statistically insignificant decline from the 4.20 mean satisfaction score for IETF 110.

    Preparedness

    Overall, preparation remains good with 83% reporting that they are either sufficiently prepared or well prepared. 4.35% were definitely under-prepared (Q8), well up on the 1.45% for IETF 110, which itself was down from 5% for IETF 109 but still a very small percentage of the total. The comments on this are mixed, with some pointing to the resources for practising with the tools, some bemoaning the lack of in-person meetings and some asking for WG chairs to provide the agenda earlier.

    Satisfaction with the agenda

    Overall satisfaction with the IETF 111 agenda was at 3.91 (Q12), a statistically insignificant decline from 4.04 for IETF 110 but now rated as 'acceptable' down from 'good'. Looking at the individual parts of the agenda (Q11), most of the satisfaction scores had no statistically significant change except for satisfaction with side meetings, which had a statistically significant improvement, and satisfaction with existing working groups, which had a statistically significant decline. Satisfaction with the opportunities for social interaction remains at 'very poor' and for side meetings, plenary and RGs it rates at 'acceptable'. For all other parts it rates 'good'.

    Satisfaction with the meeting structure

    Overall satisfaction with the structure of the IETF 111 meeting was 4.08 (Q14), a statistically insignificant drop from 4.20 for IETF 110. However, looking at the individual parts (Q13) there were statistically significant drops in satisfaction for the policy of scheduling online meetings in the timezone of the in-person meeting that they replace, for the overall length of the day and for starting at 12pm "local time", though this latter question was asked slightly differently this time. The first two are both rated as 'acceptable' and the latter is rated as 'poor' and only just above the boundary with 'very poor'. Satisfaction with 9 parallel tracks remains only 'acceptable' at 3.60 but for IETF 109 which has 8 parallel tracks, the score was 3.64 suggesting that a reduction to 8 would not improve satisfaction in any measurable way.

    Impact of being fully online

    In response to a hypothesis that moving fully online was allowing more people to participate in the IETF than would otherwise have been able to, a new question was added "If this meeting had been a normal (no COVID) in-person meeting in San Francisco with 8+ hour days and free remote participation then which of the follow best applies to you?" (Q15). Only 2 of the 148 who answered this question said that they would not have participated and 5 would have participated in fewer sessions, offset by 4 who would have participated in more sessions, suggesting that the hypothesis does not hold.

    Sessions

    41% experience no session conflicts (Q18) in IETF 111, up from both IETF 110 and IETF 109 at 35%, though a statistically insignificant increase. Satisfaction at conflict avoidance was 3.76 (Q20), a statistically insignificant increase from IETF 110 at 3.73. This again rates at 'acceptable' and suggests that more work is needed to get it up to 'good'.

    For this survey we omitted any questions on sessions running out of time to keep the survey short.

    Participation mechanisms

    Satisfaction with Meetecho remains 'good' at 4.29 (Q21) and only 'acceptable' for Gather at 3.77 and jabber at 3.68. After experimenting with Gather for four meetings we may need to experiment with alternative mechanisms for social interaction. Satisfaction with the audio streams and youtube videos both experienced statistically significant declines down to 3.84 and 4.09. Further investigation is required to understand this.

    Problem reporting

    Satisfaction with our response to problem reporting declined from 4.31 for IETF 110 to 4.00 (Q25) (the number of people responding to this question are so small that this is a statistically insignificant decline). From the comments it seems that much of this is down to confusion about what sessions require separate registration and the time it takes to fix that when someone try to join a session that is getting underway. A good improvement suggested in the text (Q26) is to put information on how to get support onto the agenda page.

    Final feedback

    Clearly, the majority just want to get back to in-person meetings with some using this survey (Q27) to complain about the decision to switch IETF 112 Madrid to an online meeting. Some are concerned that the lack of in-person meetings is making the IETF less relevant and less well supported by community volunteers.


    Share this page