Implementing the HEVC standard in software: Challenges and Recommendations for organisations planning development and deployment of software

: Implementation and use of an IT standard in software involves legal, technical and societal challenges. This paper addresses how an organisation can, and should, determine the conditions for implementation and use of the HEVC standard in software. The investigation considers the availability of the standard’s complete technical specification and the extent to which an organisation can access the information necessary to assess the licence conditions for standard essential patents impinging on the standard. Through an action case study approach the investigation analyses declarations in patent databases relevant to the standard and seeks to obtain patent licences from each declarant permitting implementation of the standard in software, where that software is to be provided under one (or several) of three specific open source software licences, and alternatively to be provided as an online service. Our analysis of legal and licensing conditions for use of the standard shows significant obstacles. We find that it is impossible to obtain licences from patent holders that would allow for implementation and use of the standard in open source software. The paper illuminates significant challenges related to conditions for use of the standard under (F)RAND terms and identifies that references to the standard in public procurement projects lead to anti-competitive effects.


Introduction
Software interoperability between heterogeneous systems presupposes lawful and technically correct1 implementations of technical specifications of IT standards.Implementation of these specifications in software projects involves a number of legal, technical, and societal challenges (Bosworth et al., 2018;Contreras, 2016;EC, 2014aEC, , 2014bEC, , 2014c;;Egyedi, 2016;Geradin, 2013;Glader, 2010;Jurata and Luken 2021;Régibeau et al., 2016).The overarching goal of the study is to investigate and establish an appropriately licensed collaborative open source software project which implements the HEVC standard2 .Since that standard is covered by a number of patents, contributors to and consumers of the project's software need to understand the licensing context so that their development and use of the software is lawful.Accordingly, the study comprises two research objectives (O1 and O2).The first objective (O1) investigates legal and licensing challenges related to conditions for implementation and use of the HEVC standard in software projects and the second objective (O2) seeks to implement the standard in a software project which is to be provided as open source software through an open collaborative platform.
This paper presents findings from addressing the first research objective which constitutes a prerequisite for establishing an open source software (OSS) project through fulfilment of the second research objective.
The HEVC standard (provided by ITU-T as ITU-T H.265 and by ISO/IEC as ISO/IEC 23008-2) is a widely deployed standard adopted in a number of different usage contexts (JCT, 2020).For example, the Swedish public broadcasting organisation SVT uses HEVC for public broadcasting of video content through SVT Play (SVT, 2021) and several organisations have adopted SaaS solutions under contract terms which require that the customers acquire patent licences covering HEVC (e.g.Lundell et al., 2021).
Research shows that it may be impossible to clarify conditions and obtain patent licences for standard essential patents (SEPs) 3 (and all necessary rights) covering specific formal standards that are provided on FRAND-terms (Lundell et al., 2015(Lundell et al., , 2019)).For example, findings from an investigation of 19 formal standards provided on FRAND-terms show it was impossible to obtain licences for all SEPs declared in the patent databases provided by ITU-T and ISO (Lundell et al., 2019).
An investigation of conditions for implementation and use of HEVC was prompted by several circumstances.First, HEVC is widely adopted and used in several different contexts (JCT, 2020;OST, 2019OST, , 2020OST, , 2021)).Second, on the date when ITU-T approved the first edition of its H.265/HEVC standard "ITU had received notice of intellectual property, protected by patents, which may be required to implement" the standard (ITU, 2013, p. ii).Third, there are several organisations which have asserted that they control SEPs that impinge on the HEVC standard through declarations in the patent databases of ITU-T4 (ITU, 2021a) and ISO5 (ISO, 2021).Fourth, the ITU-T and ISO patent databases contain declarations from several organisations that they hold SEPs relevant to HEVC, and providing the conditions under which they may be prepared to grant an appropriate licence to those SEPs.These databases contain declarations under both unclear conditions (i.e.some declarations lack details) and several declarations detailing "option 2" under the ISO and ITU-T rules implying that each such patentee "is prepared to grant a license to an unrestricted number of applicants on a worldwide, non-discriminatory basis and on reasonable terms and conditions to make, use and sell implementations" of the specific SEPs that have been declared to impinge on the HEVC standard (ISO, 2022)).Fifth, unlike many other standards the HEVC standard "has three different patent pools" (Jurata and Luken, 2021, p. 17).Sixth, findings from previous research show that it has been impossible to obtain all necessary patent licences related to the preceding ITU-T H.264/AVC standard6 (and several other MPEG standards) allowing for implementation and use of the standard in a software project which aimed to provide OSS (Lundell et al., 2019).Seventh, HEVC has been associated with "patent risks" (Zhang and Mao, 2018).Eighth, there has been several lawsuits related to the standard (EWHC, 2019;Klos, 2020;Richter, 2020).
The first research objective addresses the following research question: How can, and by which strategies should, an organisation determine and assess the conditions for implementation and use of the HEVC standard in software?
The paper presents three principal contributions.First, a strategy for how an organisation can, and should, investigate availability of the complete technical specification of the HEVC standard (section 4).Second, a strategy for how an organisation can, and should, determine and assess the conditions for implementation of the HEVC standard in software (section 5).Third, we contribute recommendations for an organisation which considers involvement with software implementations and use of the HEVC standard (section 6).

On the HEVC standard and its conditions for use
The HEVC standard has been collaboratively developed by the ITU-T Study Group 16 Working Party 3 (WP3/16) and the ISO/IEC JTC 1/SC 29/WG 11 (MPEG) Moving Picture Experts Group through the Joint Collaborative Team (JCT-VC) on Video Coding (Sullivan et al., 2012).Even though the HEVC standard is collaboratively developed by JCT-VC, it is referred to as ITU-T H.265 when provided by ITU-T and ISO/IEC 23008-2 when provided by ISO/IEC.Several editions of HEVC have been published by ITU-T and ISO/IEC (see Table 1).
HEVC is widely deployed, including in broadcast and video streaming (JCT, 2020).The contract terms for some such services frequently reveal the implementation of HEVC, including the widely deployed SaaS (Software as a Service) solution Microsoft 365 (OST, 2019(OST, , 2020(OST, , 2021)).
Further, the ITU-T H.265/HEVC standard (and the corresponding ISO/IEC 23008-2 standard) is normatively referenced in several other standards, including the sixth and seventh edition of the ISO/IEC 13818-1 standard, which in turn (via other normative references) are normatively referenced in the document file format standard ISO/IEC 29500.The H.265 standard is part of the H.26X standards family jointly developed and provided by ITU-T and ISO/IEC (ITU, 2021c;Zhang and Mao, 2018).H.265's immediate predecessor is H.264/AVC (also called ISO/IEC 14496-10), and H.266/VVC (also called ISO/IEC 23090-3) constitutes its immediate successor (ITU, 2021d;Zhang and Mao, 2018;Kerdranvat et al., 2020).Table 2 provides an overview of how these standards interrelate.Several studies have reported licensing challenges and legal disputes related to the H.264/AVC standard (e.g.Contreras, 2016;EC, 2014a;Glader, 2010;Lim, 2015;Lundell et al., 2019;Maldonado, 2014;Oliver and Richardson, 2018;Pentheroudakis and Baron, 2017).This may have implications for HEVC, especially since "HEVC/H.265introduced several new coding features in addition to AVC/H.264, such as a quad-tree-based splitting, most of the algorithms in HEVC/H.265can be traced to AVC/H.264" (Choi et al., 2020, p. 160).Hence, it may be unsurprising that the founder and initial chair of MPEG has expressed concern related to behaviour amongst non-practising entities (NPEs) that "have become more and more aggressive in extracting value from their IP" (Chiriglione, 2018).Several organisations have declared SEPs related to the standards in the H.26X family of standards (ISO, 2021;ITU, 2021a).For example, it has been reported that there "are over 2,500 patents essential to the H.264 Standard."(Maldonado, 2014) There are several organisations which assert that they control SEPs related to HEVC through declarations in (one or both of) the patent databases maintained by ITU-T (ITU, 2021a) and ISO (ISO, 2021).On 29 September 2014 the organisation MPEG LA 'announced the availability of the HEVC Patent Portfolio License ("HEVC License")' (MPEGLA, 2014a) and on 1 December 2014 the licence covered 240 patents which were controlled by 30 organisations (MPEGLA, 2014b).Amongst the 30 organisations which joined this patent pool7 during 2014 as licensors we find that for several (e.g.JVC KENWOOD Corporation, Orange SA, and Siemens Corp.) there is a lack of declarations of SEPs in the two patent databases (maintained by ITU-T and ISO) related to HEVC.Two additional (and competing) patent pools (HEVC Advance in 2015 and Velos Media in 2017) have later also been established related to HEVC (Oliver and Richardson, 2018).In January 2018 the three pools represented 23 different licensors 'with at least 50 listed assets each' (Oliver and Richardson, 2018).Each potential licensee that wishes to review draft contract terms for all three patent pools cannot do so without accepting to be bound by a non-disclosure agreement with at least one8 of these pools.The three pools (MPEGLA, 2021;Advance, 2021;Velos, 2021) represent different subsets of organisations (Oliver and Richardson, 2018;Jurata and Luken, 2021).Moreover, licensors of each pool have changed over time.For example, in January 2018 Qualcomm was one of the licensors of the Velos Media patent pool (Oliver and Richardson, 2018), whereas on 28 September 2021 it was communicated that Qualcomm ended participation in the Velos Media patent pool (Schindler, 2021).
Many organisations (including some of the organisations which have declared that they control SEPs in at least one of the two patent databases related to the HEVC) have joined one (or two) of the three patent pools that have been established in relation to HEVC.However, some organisations "expected to hold essential patents" related to HEVC have not joined any of the three corresponding patent pools (Choi et al., 2020, p. 162), including Disney, Microsoft, and Nokia as of 13 April 2021.
Concerning availability of standards, previous research identified three factors with a set of associated issues which affect "an organisation's ability to obtain and make use of the complete technical specification(s) of an IT standard" (Lundell et al., 2019).Specifically, this previous research addressed barriers which prevent establishment of software projects that seek to implement IT standards in OSS, i.e. software provided under terms that comply with the Open Source Definition (OSD, 2022).
Furthermore, previous research concerning SEP licensing identified four factors with a set of associated issues that affect "an organisation's ability to determine and assess the licence conditions under which it is able to implement an IT standard in software in the context of a software project undertaken by that organisation" (Lundell et al., 2019).
Standard-setting organisations (SSOs), such as ITU-T, ISO and IEC, do not guarantee that their respective patent database is up to date and findings show that "SSOs' patent databases may contain incomplete and misleading information" (Lundell et al., 2019).Previous research introduced the concept of apparent SEP holder when referring to an organisation which has made a declaration of SEPs related to the H.265 standard in the ITU-T's patent database (Lundell et al., 2019).An apparent SEP holder has been defined as "an organisation which fulfils at least one of the following criteria: 1.An organisation that is explicitly mentioned in patent declarations related to a specific standard included in a patent database, including organisations which have subsequently changed their name but are still the same legal entity as was mentioned in the database.
2. An organisation that has responded to a request addressed to the organisation concerning a specific standard is included in a patent database for which there is one (or several) declaration(s) that this company controls SEPs related to the specific standard.
3. An organisation which the investigator is informed by the organisation included in the patent database concerning a specific standard as declarant for particular SEPs controls those SEPs."(Lundell et al., 2019)

Research approach
For addressing the first research objective the study adopts a research method which encompasses an action-case research approach (Braa and Vidgen, 1999) for investigation of legal and licensing challenges related to conditions for implementation and use of the HEVC standard in software.Specifically, we seek to investigate how an organisation can, and by which strategies an organisation should, determine and assess the conditions for implementation and use of the HEVC standard in software.The research approach involves an iterative process of systematic data collection and review of legal conditions for implementation of the complete technical specification of the HEVC standard in software which is to be provided under conditions that fulfil (one, two or all three) specific requests related to three different licensing scenarios.
Through conduct of an action-case research approach for the investigated HEVC standard, we seek to obtain patent licences that can be used by an OSS project (and by software projects to be provided under other terms) for all SEPs which relate to all declarations in the ITU-T and ISO patent databases 9 .Specifically, the first scenario seeks to obtain 'free of charge' patent 9 Addressing the first research objective in the study presupposes that, during data collection, we can obtain and review draft patent licences in order to ensure that contract terms would allow for establishing a software project that would fulfil (at least one of) the three specific requests.It should be noted that draft patent licences could not be obtained from each patent pool related to the HEVC standard during data collection (see, for example Velos (2019Velos ( , 2021b))) since representatives for Swedish public sector organisations cannot sign any nondisclosure agreement (as under Swedish law it is impossible for a representative of a public sector organisation to guarantee non-disclosure).Further, given that this research seeks to establish an OSS project which will involve (and over time engage new) contributors there will be multiple contributors (i.e.effectively, an OSS project is based on a multi-lateral relationship between contributors to the project which will likely vary over the project life-cycle) the scope of any bilateral agreement would prevent distribution under the three specific OSS licences (in particular the GPL-3.0licence).Hence, for these reasons, conduct of our action-case research approach through which we seek to obtain patent licences would prevent us from obtaining licences from the three patent pools.
licences which allow for implementation and use of the complete technical specification of the HEVC standard in software to be provided as an OSS project under one (or several) of the three specific OSS licences GPL-3.0,MPL-2.0 and Apache-2.0(OSI, 2022).The second scenario is identical to the first scenario, except the request seeks to obtain patent licences without requesting 'free of charge' patent licences.Finally, the third scenario seeks to obtain patent licences which allow for implementation and use of the complete technical specification of the HEVC standard in software to be provided as a software project under other terms (e.g. as an online service, for example as a SaaS solution provided under the OSS licence AGPL-3.0, or as a software application provided under some other software licence, preferably comprising terms which comply with the Open Source Definition).
In this way the availability of each normative reference and declarations of SEPs in relevant patent databases were reviewed in order to identify organisations which are apparent SEP holders (related to each normatively referenced standard).
Specifically, for each declaration of SEPs in the ITU-T patent database and for each declaration of SEPs in the ISO patent database, the following request was expressed to all apparent SEP holders (on 11 June 2019) in order to address the first (and preferred) scenario: Request #1: For all patents your organisation controls which impinge on the technical specification(s) of the ISO/IEC 23008-2 standard I would like to obtain ('free of charge') patent licences for all these patents which allow for implementation of the technical specification(s) of the ISO/IEC 23008-2 standard in open source software.Specifically, I wish to obtain ('free of charge') patent licences for our collaborative research project which allow for use and implementation of the technical specification(s) of the ISO/IEC 23008-2 standard in software which is to be provided under one (or several) of the three specific open source licences GPLv3, MPLv2 and Apache 2.0.

Can you please provide copies of the relevant draft licences (related to request #1) for my consideration?
Should an apparent SEP holder be unwilling to fulfil request #1 it was also explained in the same communication that the second most preferred scenario (request #2) requested patent licences (without asking for 'free of charge' patent licences) that allow for implementation and use of the complete technical specification of the HEVC standard in software to be provided as an OSS project under one (or several) of the three OSS licences GPL-3.0,MPL-2.0 and Apache-2.0(OSI, 2022).
Should an apparent SEP holder be unwilling to fulfil both requests #1 and #2 (i.e.unwilling to provide patent licences that would allow for implementation of the standard in software to be provided as an OSS project under one, or several, of the three specific OSS licences GPL-3.0,MPL-2.0, and Apache-2.0) it was also explained in the same communication that the third (and least preferred) scenario requested patent licences that allow for implementation and use of the complete technical specification of the HEVC standard in software.Hence, this would imply that software would be provided as a software project under other terms.In acknowledging that the ISO and ITU-T rules do not specifically address OSS, the reason for including request #3 was motivated by our desire to implement and use the HEVC standard in software which is to be provided as an online service (e.g.under the OSS licence AGPL-3.0), or to be provided under other conditions (preferably under terms which would comply with the Open Source Definition).A further motivation for the third scenario was to consider if, were the existing approved OSS licences unacceptable, it would be possible to draft a new software licence that we consider would comply with the Open Source Definition and can therefore legitimately permit the software developed by the project to be provided as OSS 10 .
Concerning the formulation of the three requests (which included a reference to ISO/IEC 23008-2, as detailed above), the letter of the request sent to each apparent SEP holder made it clear that ITU-T H.265 is the same standard.
The data collection process involved an extensive dialogue11 over a long time-window12 , in many cases involving several reminders and clarifications provided in subsequent dialogue with different representatives13 for organisation which have declared SEPs in the ITU-T and ISO patent databases related to the HEVC standard.For example, during several dialogues with representatives for organisations which have made declarations under "option 2" (ISO, 2022) discussions involved willingness to provide a paid licence under a lump sum that would allow for unlimited redistribution of OSS from an OSS project to be provided under the GPL-3.0,MPL-2.0, and Apache-2.0licences.

Availability of the complete technical specification of the standard
Different editions of the HEVC standard have been provided by the ITU-T (as the ITU-T H.265 standard) and the ISO/IEC (as the ISO/IEC 23008-2 standard).The HEVC standard has changed over time and also contains normative references, that is references to other standards.Access to those normatively referenced standards is necessary when using and implementing the relevant standard.
Based on previous research (Lundell et al., 2019), this section presents findings relating to the availability of the complete technical specification of the HEVC standard focusing on its normative references and implementations of the HEVC standard in software.
A complete implementation of the HEVC standard requires that all normatively referenced standards are also implemented.In turn, this means that the technical specification of all those normatively referenced standards also must be available.In addition, there are several levels of normative references (i.e. the HEVC standard references other standards, which in turn reference other standards, and so on).Implementing HEVC requires access to all of the standards normatively referenced within the HEVC standard, both directly and indirectly.
The fourth, fifth, sixth, seventh, and eighth editions of ITU-T H.265 each include eight direct normative references (the same eight, in each case).All normatively referenced standards in those editions are undated, implying that for each one the latest edition of each referenced standard (including any amendments) applies.The importance of establishing a collaborative software project for supporting development of H.265 was recognised early by the JCT-VC team.The report of the first JCT-VC meeting (held 15-23 April 2010) states that a reference software "for the standard shall be provided with a suitable copyright disclaimer header text in a form acceptable to the parent bodies to enable publication of the source code and to enable users of the software to copy the software and use it for research and standardization purposes and as a basis for the development of products" (JCT, 2010a, p. 4).
At the first JCT-VC meeting, held on 16 April, collaboratively developed source code provided under the BSD 3-Clause licence (marked "Copyright (c) 2010, SAMSUNG ELECTRONICS CO., LTD. and BRITISH BROADCASTING CORPORATION") was presented for software which implements the (then current) draft H.265 standard (JCT, 2010a).Further, report of the second JCT-VC meeting (JCT, 2010b) and also a project management report (JCT, 2010c) states that the "intent is for the software to be developed as part of the work to develop the HEVC standard and also for it to be published as reference software by ITU-T and ISO/IEC" (JCT, 2010b(JCT, , 2010c)).It was also concluded that the BSD 3-Clause licence (a software licence approved by the OSI), approved by the WG11 parent body (in July 2009), may be appropriate (JCT, 2010b(JCT, , 2010c)).Technical discussions and appropriate licensing related to the reference software has been ongoing issues for discussion as indicated in the meeting report of the third JCT-VC meeting.This states that "the software copyright management should be established in a manner that protects the patent rights of contributors (which are subject to the ITU-T/ITU-R/ISO/IEC Common Patent Policy)" (JCT, 2010d).Further, the report also highlights "a further desire to ensure full clarity that no patent rights are granted by the availability of the software" (JCT, 2010d).
However, the meeting report of the fifth JCT-VC meeting clarifies that the reference software for the standard will not be provided under the BSD 3-Clause license approved by the OSI (2022) but rather an unapproved adapted version of it, which explicitly states that no patent rights are granted for the reference software (JCT, 2011a).This conclusion therefore implies that the reference software is not to be provided under an OSS licence (since the reference software is provided under a software licence that fails to comply with the Open Source Definition and the licence is also not approved by the OSI).This is reinforced by the project management report from the same JCT-VC meeting which states: "The copyright in this software is being made available under the BSD License, included below.This software may be subject to other third party and contributor rights, including patent rights, and no such rights are granted under this license."(JCT, 2011b) Besides the challenges in developing a precise, correct, and complete technical specification of the H.265 standard, inclusion of undated normative references in each edition of the specification of the standard imposes additional complexity for software projects seeking to implement a specific edition of H.265.Each edition of the H.265 technical specification includes undated normative references which cause the complete technical specification of each edition of the H.265 standard to become a moving target.Because undated normative references are used, the complete technical specification of each edition of the standard will change several times after the date for publication of each edition.This is because the set of normatively referenced standards will be different over time.
This becomes even more complex for a software project which seeks to address software interoperability and longevity of files.This occurs because a specific implementation must implement several specific editions of the complete technical specification for H.Over time, H.265 has been implemented by several software projects resulting in a range of applications, products and services (JCT, 2020).A report from a survey conducted by the JCT-VC team which investigated deployment status for H.265 shows that there are OSS projects that have implemented the standard (JCT, 2020).

Ability to determine and assess the licence conditions for SEPs impinging on the standard
Before lawful use and implementation of HEVC in software, an organisation needs licences to each of the patents (standards essential patents -SEPs) which impinge on that implementation.
Our findings reveal issues arising with the process of obtaining (or trying to obtain) licences, including challenges in locating, opening communication and having meaningful dialogue with each apparent SEP holder.
This section presents our findings relating to an organisation's ability to determine and assess the licence conditions enabling it to use and implement HEVC in software.Based on previous research (Lundell et al., 2019) which identified four factors affecting SEP licensing with associated issues (detailed as twelve questions), we investigate ability to determine and assess the licence conditions for SEPs impinging on the standard through an analysis of these twelve issues (as detailed by Lundell et al., 2019).Each factor with its associated issues has been investigated and is presented in separate subsections.

Ability to reach all organisations behind SEP declarations related to the HEVC standard
Attempts were made to write to each organisation which on 11 June 2019 was listed as declaring SEPs in one or both of the two patent databases provided by ITU-T (covering ITU-T H.265) and ISO (covering the corresponding ISO/IEC 23008-2 standard) by both postal mail (air-mail) and email.Many led to no response, or returned postal mail or notification of undelivered email.
Based on returned letters we find that several addresses listed in the databases were no longer current.Overall, letters (sent via postal or email) related to more than one out of three addresses listed for the patent databases were either returned as undelivered (in the case of postal mail), or generated an email non-delivery notice.
In cases where a delivery attempt failed (either email or postal address) an effort was made using online databases (a web search, company registries, and annual reports) to find an appropriate address for that declarant.In several cases, no response to our letters or emails was received from the organisation to which they were addressed, but neither did we receive an indication that the letter was undelivered.

Ability to obtain responses to requests for patent licences from apparent SEP holders
The vast majority of the apparent SEP holders acknowledging receipt of the letter failed to respond to the three specific requests.Instead of responding to the requests, several of those organisations instead expressed (in general terms) that they were willing to provide licences under RAND terms for a fee 'per unit'.Several of those organisations15 also referred to one (or two) of the three patent pools established in relation to the HEVC standard.For example, one apparent SEP holder referred to one of these patent pools and provided the following response to the letter: "Right now the only option I am aware of is to obtain a royalty-bearing license from MPEG LA".
Further, another apparent SEP holder which referred to another of the three patent pools clarified the implications of being bound by a RAND obligation as follows: "Since we have a RAND (reasonable and non-discriminatory) licensing obligation for our relevant patents, we are not in a position to provide you with royalty free licenses (your Request #1).This is because any such arrangements would be unfair and discriminatory to others who have licensed our relevant patents and pay per unit royalties through the MPEG LA HEVC patent pool program." One fourth of the organisations contacted 16 responded by referring to one (or several) of the three patent pools established to license HEVC (MPEG LA, HEVC Advance and Velos Media) instead of responding to the three specific requests.The majority of them referred only to one of the three patent pools, but several also referred to two.One organisation even referred to all three patent pools (interestingly, that respondent was only affiliated with one of the three pools).
A minority of the organisations contacted responded to the letter within three months (even after a reminder).Of those, the majority were unable or unwilling to provide the requested licences.For example, one organisation commented (in response to a reminder): "we are awfully sorry to inform you that we cannot help you." Several organisations expressed that they were unable or unwilling to provide draft licences.Some organisations also stressed the need to sign a non-disclosure agreement before engaging in a bilateral negotiation.For example, one apparent SEP holder expressed that "license agreements involve a wide range of negotiated terms and conditions that are dependent upon numerous economic, business, legal and other considerations with respect to both licensee and licensor.Such terms and conditions are virtually always considered during the course of highlydetailed bilateral discussions under a negotiated mutual non-disclosure agreement, under common industry practice." Similarly, another organisation required signature of a non-disclosure agreement before providing any draft licence agreement: "before we provide any of our confidential information, including our license agreement, we require prospective licensees to enter into a non-disclosure agreement".Obviously, for any representative of a planned collaborative OSS project which is to be provided on an open collaborative platform it is impossible to sign any such nondisclosure agreement17 .
Amongst responses, one organisation also stated that the licence for their SEPs is not designed for source code to be developed in software projects.In the words of this organisation: "Up to now, our license offer is intended for HEVC codecs ready to work in end-user products, not for any source codes distributed under OSS licenses.… I'm sorry that we can't accept either of your requests." Further, some organisations also provided a response with a proposal to offer a licence with restrictions on use, including a limitation on implementation for 'research purposes', something which clearly would inhibit implementation in software to be provided under any OSS licence, including the licences in the requests.

Clear declarations of SEPs related to the HEVC standard in patent declarations
Each parent body for the joint JCT-VC team maintains its own patent database declarations from apparent patent holders (41 organisations in total 18 ) for the HEVC standard.Consequently, it is necessary to analyse the content of each patent database and for this reason we sent the request to each organisation that had provided a declaration related to the standard in at least one of the two patent databases maintained by ITU-T and ISO.
Declarations (in the patent databases) from several apparent SEP holders related to the HEVC standard lack details of the specific SEPs for which they are claiming rights.Even though permitted by the SSO's rules, this lack of reference to a specific patent (by patent number, for example) makes legal analysis of declarations challenging.For example, one apparent SEP holder responded to the request for licences with the following request to identify which specific patents the request concerned: "[Company name] has over 25,000 patents, and they are not organized according to whether the are standard essential or not, so some identification of the relevant patents would be necessary".Given that the content in the patent databases from this apparent SEP holder lacked the necessary details concerning the relevant patents (and since apparent SEP holders also seemed to lack this information) identification of the relevant patent became difficult (if not impossible) for any organisation seeking a licence.
There are inconsistencies between the content of the two patent databases for the HEVC standard.On 11 June 2019, ITU-T's patent database contained declarations from 37 different organisations, whereas the ISO's contained declarations from 31 different organisations.27 declarations were in both databases.In some cases, a declaration was found against the standard in one database, but not the other.
Note also that the content of each patent database provided by ITU-T and ISO is guaranteed to be neither accurate nor complete.Each SSO cannot require anyone not involved in a standard setting process to disclose patents (even though the SSOs encourage apparent SEP holders to provide declarations for inclusion in each patent database).Further, each SSO does not verify that the entries which are in the patent database actually do impinge on the H.265 standard.

Provision of licences that allow for implementation of the HEVC standard in OSS projects
The investigation clarified the intention to establish an open collaboration through an OSS project which uses and implements the complete technical specification of the HEVC standard (and all its normative references) in software that is to be provided under three specific OSS licences: GPL-3.0,MPL-2.0 and Apache-2.0(as detailed in request #1 and request #2).Further, the investigation also clarified that the intention is to develop, maintain and distribute the software on an open collaborative platform under one (or several) of these three specific licences.
Each declarant that provided a response to the requests was unwilling to provide a licence compatible with the specific OSS licences under which the OSS project was stated to be licensed to users and recipients of the developed software.Hence, it follows that request #1 cannot be fulfilled.
Most organisations which expressed that they are willing to provide a licence on RAND terms referred to one (or two) of the patent pools.For example, one apparent SEP holder stated: "Of course, we could grant the license to you bilaterally, however, in general, bilateral license royalty would be higher than pool royalty rate because we need to add our cost for negotiation.Therefore, we respectfully recommend you to consider taking license from HEVC Advance."Some apparent SEP holders suggested, or requested a discussion by phone.Clearly, some of the organisations had a genuine interest in trying to better understand the requests.For example, during one such dialogue (in February 2020) both parties engaged in a constructive dialogue concerning the planned OSS project, which included discussions related to the motivation and need for clarifying legal and licensing conditions for the HEVC standard before initiating the planned subsequent software development activities.During the discussion it was suggested by the patent holder that they will come back within a few months with a proposal and draft licence, something which never happened.In two other cases, apparent SEP holders proposed a phone call (which was agreed in each case), but neither meeting took place.In one case, on 6 February 2020 after some email dialogue a call was agreed and scheduled for 24 February 2020.However, late in the evening on 18 February 2020 the declarant cancelled the meeting.In another case, despite an agreed meeting scheduled well in advance, no representative for the declarant dialled in.The same thing happened at an agreed rescheduled meeting.After these events, neither declarant has responded to further reminders.
A number of declarants who provided a response expressed willingness to provide a licences under RAND terms.It is clear from those responses that each such offer (with one notable exception, see below) assumed a payment 'per unit'implying that request #2 cannot be fulfilled.An inherent characteristic of the OSS licensing model is that software may be freely distributed (in a cascade) to multiple recipients, making it impossible to control the number of copies made.Consequently, any licence for an SEP that impinges on HEVC requiring a payment 'per unit' is inherently incompatible with the three specific OSS licences (and in addition also incompatible with all other OSS licences which fulfil the Open Source Definition).
Amongst apparent SEP holders which offer their SEPs on royalty-bearing RAND terms there is one organisation which in its initial (and so far only) response (provided within seven weeks) indicated a willingness to provide a licence for their SEPs on RAND terms for an unlimited number of copies (i.e. a 'fixed amount') it was clarified that the royalty amount would be 'impractically' large.In the words of this apparent SEP holder: "As to your Request #2 and #3, we are ready to license those patents under FRAND conditions if we could receive reasonable per-unit royalties which are calculated based on the number of the copies of the software being distributed.However, under the situation that, as you have explained, it is impossible to trace the accurate number of the copies of the software being distributed, we consider that it is difficult for us to license our patents.(It is possible for us to consider to license our patents for unlimited number of copies of the software.However, in such case, the royalty amount which we would ask you to pay would be impractically large for you.)" Further, as this organisation not responded to a request for draft licence terms it follows that the actual price is unclear and also if the offered terms would be compatible with the specific OSS licences.
Consequently, it follows that the investigation has been unable to identify any specific licence from any apparent SEP holder which would allow for implementation of the HEVC standard in software that is to be provided as OSS.
After concluding that both request #1 and request #2 cannot be fulfilled, it was decided to abandon the initially planned investigation of the complete technical specification of the HEVC standard including all standards normatively referenced within.

Analysis and Implications
According to the results of the investigation, the study finds major legal and licensing challenges related to conditions for implementation and use of the HEVC standard in software projects.Based on an extensive investigation and analysis of information provided by organisations which have asserted (to the relevant SSOs) that they control SEPs related to the HEVC standard and by analysis of information provided by SSOs, findings show that an organisation which aims to establish and maintain a software project that implements the complete technical specification of the specific standard has been unable to obtain licences for the standard that would allow for lawful use and implementation in software.In summary, responses (and lack of responses during the investigation) from apparent SEP holders show that it has been impossible to obtain requested licences (under any of the three different requests), despite allowing a very long time frame (more than 3 years) for obtaining a response from each apparent SEP holder related to each of the three specific requests.
In particular, analysis of responses from apparent SEP holders during the investigation also shows stark evidence to suggest that, under current legal and licensing conditions related to use of the specific standard, it is impossible to lawfully establish a collaborative OSS project which seeks to implement the complete technical specification of the standard.Further, findings from the study show a number of additional (technical, organisational, and societal) challenges to be successfully addressed to allow the lawful establishment of a software project on an open collaborative platform that implements and uses the standard and provides developed software under an OSS licence.
From the extensive and persistent dialogues with all responsive apparent SEP holders concerning an organisation's ability to determine and assess the licence conditions for SEPs impinging on the complete technical specification of the HEVC standard, we find major challenges which prevent the organisation from obtaining all necessary licences from all apparent SEP holders.Specifically, from attempts to establish a dialogue with apparent SEP holders which have made declarations in the relevant patent databases we find that the majority of apparent SEP holders are unresponsive to requests.We elaborate on four specific observations.
First, findings from the study show significant uncertainty and lack of responses from apparent SEP holders.Despite extensive attempts to establish a dialogue with each apparent SEP holder after several reminders (and within 38 months from sending the initial requests), we note that for a majority of apparent SEP holders these attempts to obtain a response were unsuccessful.Further, several apparent SEP holders suggested (or requested) a phone dialogue related to the requests, which we agreed to in each case.However, we find that none of these cases led to a response.In addition, each apparent SEP holder (amongst the minority of organisations) which provided a response to the requests clarified an unwillingness to provide a licence that will fulfil the requests.
Second, to support development of the first edition of the HEVC standard the JCT-VC team initiated efforts for development of a reference software for the standard.Early contributions to development of the reference software were made by representatives for individual organisations which therefore held the copyright to their specific contributions, but made them available under a software licence that does not qualify as an OSS licence as its terms do not fulfil the Open Source Definition.The reference software is provided under a software licence (based on the BSD 3-Clause licence) which explicitly excludes patent rights.For example, one source code file19 contributed by two organisations (Samsung and BBC) to the reference software contains the following terms: "This software may be subject to other third party and contributor rights, including patent rights, and no such rights are granted under this license.
Copyright (c) 2010, SAMSUNG ELECTRONICS CO., LTD. and BRITISH BROADCASTING CORPORATION All rights reserved." Third, during dialogues with apparent SEP holders we find that several organisations refer to one (or two) of the three patent pools established to cover HEVC.Based on analysis of the information provided by these patent pools we find that there are several apparent SEP holders which appear as licensors in one (or several) of these patent pools for the HEVC standard, despite the fact that we have been unable to identify any declaration by the same organisations in any of the two patent databases (provided by ISO and ITU-T).Consequently, any analysis of apparent SEP holders needs to go beyond the content of the patent databases provided by the relevant SSOs.
Fourth, based on analysis of available documentation from the JCT-VC meetings (40 meetings in total) a number of organisations have been identified as contributors to development of the H.265 standard and at the same time identified as one being of the licensors in a patent pool, whereas we have been unable to identify any declaration by the same organisations in the two patent databases (provided by ISO and ITU-T).Hence, it follows that any analysis of apparent SEP holders must also include other organisations that may (or may not) have contributed to development of the standard even if there are no declarations in any of the patent databases provided by the relevant SSOs from those organisations.
Findings show that it has been impossible to obtain responses from the vast majority of apparent SEP holders within 16 weeks, something which highlights challenges for EU law for public procurement that for a long time has aimed to stimulate a fair and competitive market based on important principles of transparency, non-discrimination and equal treatment (Directives 2004/17/EC and2004/18/EC;Glader, 2010).These (and subsequent versions of these) EU directives clarify how technical specifications can and shall be used in public procurement processes.Further, amongst the minority of apparent SEP holders which provided a response within 16 weeks we find that each of these either referred to one (or two) of the patent pools, or clarified that they were unable to fulfil any of the three requests.
There are a number of, potentially very problematic implications of an inability to obtain licences for the HEVC standard which allow for implementation and use of the standard in a software project.Below we elaborate five such implications.
First, the HEVC standard includes normative references which in turn (in a cyclic manner) include the same standard.Specifically, the ISO/IEC 23001-11 standard includes several normative references, including the undated ISO/IEC 23008-2 standard, the undated ISO/IEC 14496-10 standard, and the undated ISO/IEC TR 23009-3.Further, it follows that all editions of the ISO/IEC 23008-2 standard (and all corresponding editions of the ITU-T H.265 standard) are normatively referenced from the ISO/IEC 29500 standard (via other standards) and that several normatively referenced standards refer to each other 'in a cyclic manner' (e.g. the two standards ISO/IEC 23001-11 and ISO/IEC 23008-2 refer to each other).Hence, it follows that any comprehensive analysis of conditions for licensing of the complete technical specification of any standard must also include an analysis of all relevant parts, versions, and editions of all (directly and indirectly) normatively referenced standards.
Second, the HEVC standard is deployed in a number of widely used software applications, including the SaaS solution Microsoft 365.One of the contract documents expresses that a customer which uses Microsoft 365 must obtain its own licences for HEVC: "Customer must obtain its own patent license(s) from any third party H.265/HEVC patent pools or rights holders before using Azure Media Services to encode or decode H.265/HEVC media."(OST, 2019) Findings from the present study show stark evidence to suggest that it may be impossible for any organisation to obtain such patent licences for the HEVC standard, and it follows that it may be unsurprising that many organisations that have agreed to such terms also have failed to successfully obtain such patent licences (Lundell et al., 2021).
Third, the HEVC standard constitutes an inherent part (as a normatively referenced standard via other standards) of the widely referenced file format ISO/IEC 29500 standard.Under the assumption that this file format has been implemented by Microsoft in a range of versions of its Office applications (some time after the 'docx-format' was first introduced via the third service pack related to the Microsoft Office 2003 suite) it follows that a large set of 'docx-files' Fourth, the HEVC standard includes undated normative references which implies that the complete technical specification of the standard will evolve even if the 'top level' specification of a specific edition of the HEVC standard is unchanged.Accordingly, any legal analysis of conditions for use of the standard needs to be an ongoing process.This imposes challenges for maintenance of a software project which seeks to faithfully maintain a lawful and technically correct implementation of all editions of the complete technical specification of the ITU-T H.265 standard in a software application.Specifically, the undated ISO/IEC 23008-2 standard is included as a normative reference in the second edition of the ISO/IEC 23001-11 standard, which in turn is included as an undated normative reference in the ninth edition of the ISO/IEC 14496-10 standard that was published during December 2020.Further, the undated ISO/IEC 14496-10 standard constitutes a normative reference in the first edition of the ISO/IEC 14496-15 standard, which in turn is included in fourth edition of the ISO/IEC 14496-1 standard.
Findings from previous research which investigated the possibility to obtain licences for the ISO/IEC 14496-10 standard (i.e. the H.264/AVC standard) show that several apparent SEP holders for this standard have been unresponsive and that it may be impossible to obtain licences for this standard that allow for implementation in a software project (e.g.Lundell et al., 2019).Further, previous studies show that the H.264/AVC standard has been subject to legal disputes (e.g.Contreras, 2016;EC, 2014a;Glader, 2010;Maldonado, 2014;Pentheroudakis and Baron, 2017) and findings show that it may be impossible to obtain all necessary licences for the standard to allow for implementation in software projects (Lundell et al., 2019) and it is also clear that there are close links between the H.264 and H.265 standard.For example, it has been stated that even though the "HEVC/H.265introduced several new coding features in addition to AVC/H.264, such as a quad-tree-based splitting, most of the algorithms in HEVC/H.265can be traced to AVC/H.264."(Choi et al., 2020, p. 160) Fifth, based on analysis of attempts to establish dialogues with all apparent SEP holders and information provided by the subset of apparent SEP holders which provided a response, we find that it is highly unlikely that any software projectunder any circumstanceswould be able to obtain all patent licences that would (lawfully) allow for establishing a software project which implements and uses all editions of the complete technical specification of the HEVC standard.Further, any attempt to establish a software project which seeks to provide OSS (or proprietary software) from a software project must, at least, obtain patent licences from all apparent SEP holders (that have made declarations in the two specific patent databases provided by ITU-T and ISO) and all licensors that have joined one (or several) of the three patent pools established in relation to the HEVC standard (and all its normatively referenced standards, i.e. all apparent SEP holders and all licensors in any of the relevant patent pools related to all those normatively referenced standards).In addition, as clarified by responses from some apparent SEP holders, a licence from a patent pool is designed for a bilateral scenario.Hence, we find that such licences are unsuitable for a collaborative software project which aims to provide OSS.Consequently, it follows that any such licence cannot fulfil any of the three specific requests.
In acknowledging that each declaration (in the two relevant patent databases provided by ITU-T and ISO) made by each apparent SEP holder clarifies that each organisation is unwilling to provide a royalty-free licence, we note that each organisation states that there is an intent to negotiate a licence under (F)RAND terms.However, findings from the dialogues with apparent SEP holders clarify that each organisation is only willing to provide a licence based on per unit royalties calculated by numbers of such units provided from each licensee.Further, the investigation also experienced requests for confidentiality during negotiations in relation to licences for the HEVC standard, something which is common practice amongst apparent SEP holders that have made (F)RAND commitments according to a previous study which claims that "licensing agreements often contain confidentiality clauses" (Régibeau et al., 2016).Given that the specific usage scenario (as detailed in the requests) makes it is impossible to count how many copies that finally will be created and implemented in software it follows that a lump sum payment arrangement would be the only possible option.Consequently, we find that the (F)RAND terms offered by apparent SEP holders do not fulfil any of the three requests.
In summary, findings from the investigation of the HEVC standard show that contemporary IPR policies used by SSOs (which allow for use of RAND terms) inhibit implementation of the standard in OSS projects.Hence, results from the study illuminate a practice which shows inherent inconsistencies between work practices used by apparent SEP holders and conditions for use of standards by software projects which seek to provide OSS.

Discussion and Conclusions
Findings from the study show that the HEVC standard provided under FRAND-terms causes significant obstacles for organisations that seek to use the standard.In particular, findings from the investigation of legal and licensing conditions for implementation and use of the HEVC standard show that it is impossible to obtain licences from apparent SEP holders that would allow for lawful use of the standard.This is particularly problematic from a public procurement perspective and for attempts to establish a software project which seeks to implement and use the HEVC standard and which is to be provided as OSS.
Where a public sector organisation wishes to implement such a standard in software (for example, to ensure that the digital assets which it holds in a specific format remain capable of being lawfully obtained, processed, and distributed when a proprietary solution making use of that format is no longer available), it will require relevant patent licences, which it must either obtain itself, or have obtained for it by a third party, under a lawful procurement process.
Findings from the study shows that for the investigated standard (HEVC) it was impossible to access all relevant contractual licence terms without signing an NDA.Since a Swedish public sector organisation (under Swedish law) is unable to sign any NDA we find that it would be unlawful and inappropriate for a public sector organisation to enter a contract which would imply having to accept unknown contract terms.From a practical perspective, note also that given the effort applied by the researchers to obtain such a licence, and the unwillingness of apparent SEP holders to engage appropriately, it seems unlikely that any reasonable procurement process would have permitted an appropriate licence to be granted within the time permitted by such a procurement process, or at all.
The investigation showed stark reluctance amongst several apparent SEP holders to engage in a dialogue related to the three requests, with the goal of clarifying conditions for use of the HEVC standard remaining unreached.This may indicate a significant difference in traditions between different stakeholder groups.In particular, the study highlighted different business interests amongst apparent SEP holders and IPR policies used by formal standards setting organisations.Further, an effort to first clarify conditions for use of the complete technical specification of the HEVC standard before starting to implement and use the specific standard was, amongst some apparent SEP holders, seen with suspicion and considered unusual.Some apparent SEP holders even requested a signed non-disclosure agreement before they were willing to initiate a discussion.
Findings from the study clearly show that amongst apparent SEP holders there are widespread conceptions that bilateral negotiations over terms for a licence and royalty payment 'per copy' is seen as a norm.Consequently, the requests related to the aim of establishing a software project which seeks to implement the complete technical specification of the HEVC standard that will provide software under one (or several) modern OSS licences seems very far from what can be accepted by any apparent SEP holder.In particular, a royalty payment without a cost 'per unit' (related to request #2) was considered unrealistic by many apparent SEP holders.It may therefore be unsurprising that no organisation is willing to provide a licence that would allow for fulfilling any of the three specific requests.
In retrospect, we would characterise the approach taken by several apparent SEP holders in their reactions to the attempts to establish an effective dialogue related to the requests during the investigation as an 'avoidance strategy', in the sense that such organisations wished to appear as willing to provide a licence, where in fact the actual actions undertaken by each such organisation demonstrated the opposite.
Funding: This research has been financially supported by the Swedish Knowledge Foundation (KK-stiftelsen) and participating partner organisations in the SUDO project.
Informed Consent Statement: Conduct of the study utilised an action case research approach through which we requested licences from apparent SEP holders that publicly have made declarations in public patent databases related to a specific standard that we planned to implement by establishing a collaborative software project for provision as open source software.
265.For example, a software project which during May 2018 or May 2019 aimed to implement the complete technical specification of H.265 needed to implement eight different editions of the standard, including all amendments and corrections.Five were published by ITU-T and three by ISO/IEC.The project also needed to include all directly and indirectly normatively referenced standards.For example, an implementation during May 2018 of the complete technical specification of the fifth edition of the H.265 standard (provided by ITU-T) includes (via its undated normative reference ISO/IEC 23001-11) the need to implement the ISO/IEC 13818-1:2013 standard (i.e. the fourth edition of ISO/IEC 13818-1).This is because the fifth edition includes ISO/IEC 13818-1 as a normatively referenced standard.However, an implementation of the same edition of the H.265 standard during May 2019 included the need to implement the latest edition of the ISO/IEC 13818-1 standard.By May 2019 the latest edition was the sixth edition 14 of the ISO/IEC 13818 standard.This occurred because, unlike the first edition of the ISO/IEC 23001-11 standard, the second edition of the ISO/IEC 23001-11 standard included an undated normative reference to ISO/IEC 13818-1, while the first edition contained a dated normative reference.This illustrates that the complete technical specification of the fifth edition of the H.265 standard has changed between May 2018 and May 2019.This is the case even though the technical specification of the fifth edition of the H.265 standard as provided by ITU-T was itself unchanged.
have been produced over the years after the publication of the different parts and editions of the ISO/IEC 29500 standard.Specifically, the second edition of the ISO/IEC 23008-2:2015 standard (which corresponds to the second edition of the ITU-T H.265 standard) is a normative reference in the sixth edition (published March 2018) and seventh edition (published June 2019) of the ISO/IEC 13818-1 standard.In turn, the undated ISO/IEC 13818-1 standard is a normative reference in the fourth edition of the ISO/IEC 13818-7:2006 standard, which in turn is a normative reference in the fourth edition of the ISO/IEC 14496-1:2010 standard.Further, the undated ISO/IEC 14996-10 standard is normative referenced in the first edition of the ISO/IEC 14496-18 standard.In addition, the undated ISO/IEC 14496-18 standard is a normative reference in the first edition of the ISO/IEC 14496-22:2007 standard, which in turn is a normative reference in the first edition of part 1 of the ISO/IEC 29500 standard.Similarly, the undated ISO/IEC 14496-18 standard is a normative reference in the second edition of the ISO/IEC 14496-22:2009 standard, which in turn is a normative reference in the second, third, and fourth editions of part 1 of the ISO/IEC 29500 standard.Hence, it follows that the H.265 standard (indirectly, via several other normatively referenced standards) constitutes an inherent part of the ISO/IEC 29500 standard.

Table 1 :
Overview of the published editions of the HEVC standard

Table 2 :
Overview of the H.26X family of standards Each edition of the H.265 standard published from 13 April 2013 (first edition) to 22 August 2021 (eighth edition) includes the undated ISO 12232 standard as a normative reference.The ISO 12232 standard was published in its second edition on 15 April 2006 and subsequently published in its third edition during February 2019.At the time of publication for the fifth edition of the H.265 standard (on 13 February 2018) the latest edition of the undated normatively referenced ISO 12232 standard was the second edition, whereas during February 2019 the third edition of the ISO 12232 standard became the latest edition of this normative reference.Therefore, the complete technical specification of the fifth edition of the H.265 standard changed during February 2019 because the third edition of the ISO 12232 standard at that time became the latest edition of this undated normative reference.
The second edition of the ISO 12232 standard contains five undated normative references (provided by ISO, IEC, and ITU-R), whereas the third edition of the ISO 12232 standard contains three normative references (provided by ISO and ITU-R) of which two are undated and one is dated.This shows that the complete technical specification of a specific edition of the H.265 standard may change due to publication of new editions of normative references, even if the technical specification of the H.265 standard (at the top level) is unchanged.