Tag: ANN

Concept terms scoring/GOOGLE – T 0872/19 – 14 October 2021

In this decision, the appellant stated that a “web page” is a physical entity, corresponding to an “image”. The Board has considerable doubts about this. In fact, the inventive step did not depend on this.

Object of the Invention:

  • the subject matter relates to online advertisement auctions
  • when an online advertisement auction is to be conducted to select one or more advertisements to be included in a resource/ web page, resource features are extracted from the web page
  • a concept term scoring system uses the received resource features to predict a vector of scores that includes a score for each of the set of concept terms/ advertising features
  • in online advertising, the concept term scoring system can generate a score for each of a set of concept terms that may be used as advertising keywords for selecting advertisements for participation in the auction
  • the concept term scoring system includes a deep network and a classifier
  • Claim 1 differs from the closest prior art inter alia in that it is configured to process the alternative representation of the input to generate a respective relevance score for each concept term in a pre-determined set of concept terms, wherein each of the respective relevance scores measures a predicted relevance of the corresponding concept term to the resource

Appellant I (inventive step – 3rd auxiliary request)

  • the objective problem to be solved was to provide a more accurate representation of the resource features by generating an alternative representation of these resource features “in a new way”
  • in decision T 208/84, point 3, the board found that a method for image processing was susceptible of industrial application
  • T 208/84 stated the following: “Clearly a method for obtaining and/or reproducing an image of a physical object or even an image of a simulated object (as in computer-aided design/computer-aided manufacturing (CAD/CAM) systems) may be used e.g. in investigating properties of the object or designing an industrial article and is therefore susceptible of industrial application. Similarly, a method for enhancing or restoring such an image, without adding to its informational content, had to be considered as susceptible of industrial application.“
  • in T 208/84, points 6 and 7, the board stated that it was of the opinion that “even if the idea underlying an invention may be considered to reside in a mathematical method a claim directed to a technical process in which the method is used does not seek protection for the mathematical method as such” and concluded that “[i]n contrast, a ‘method for digitally filtering data’ remains an abstract notion not distinguished from a mathematical method so long as it is not specified what physical entity is represented by the data and forms the subject of a technical process, i.e. a process which is susceptible of industrial application“
  • thus, the board equated a “technical process” with “a process susceptible of industrial application“
  • in accordance with the case law, “processing images” was considered as being susceptible of industrial application and thus was technical
  • according to the appellant a “web page” is a “physical entity”

Board I (inventive step – 3rd auxiliary request)

  • the board agrees that the electrical signals producing the “web page” are physical entities and that a web page printed on a piece of paper becomes physical
  • but it has strong doubts that a “web page” can be considered as a “physical entity“
  • rather, a “web page” is data representing information

Appellant II (inventive step – 3rd auxiliary request)

  • it would thus be correct to consider processing other entities, such as text, to be technical as well
  • the appellant questioned the difference between processing images versus processing text: the words in a particular language had a meaning (except for nonsensical words or sounds) that a (sufficient) number of people understood in the same manner as a (sufficient) number of people recognised a particular colour as being “red”, for example, even though this understanding was not “universal”
  • both an image and a web page were physical entities and that the claimed system concerned the processing, i.e. the analysing and classification, of physical entities
  • the classification of images had been recognised in the case law as solving a technical problem

Board II (inventive step – 3rd auxiliary request)

  • the board did not consider whether the system of claim 1 might be susceptible of industrial application but indeed whether the system had a technical effect going beyond the mere implementation of a non-technical method on one or more computers
  • however, providing a more accurate “alternative representation” of the resource features, and therefore refined relevance scores, is not a technical effect since the relevance scores do not constitute technical features
  • this does not depend on whether the “resources” are text, web pages, images or multimedia content, as in page 1, lines 4 to 5, of the description, and thus does not depend on whether the “resource features” are features of text, web pages, images or multimedia content (G 1/19 of 10 March 2021, Reasons 126, last sentence; see also T 1924/17 of 29 July 2019, Reasons 12 to 13).

Board III (inventive step – 3rd auxiliary request)

  • the relevance scores measure a predicted relevance of the “concept terms” to the resource
  • taking the example of an “image” as a “resource”, the concept term might be “cat“
  • the (first) relevance of this “concept term” to a first image comprising only dogs might be lower than its (second) relevance to a second image comprising exactly one dog and one cat, which might itself be lower than its (third) relevance to a third image comprising two cats or more, as an example
  • inversely, the three images exemplified here might be classified by their relevance to the concept term “cat”: the third image is more relevant than the second image, which is itself more relevant than the first image
  • the system of claim 1 does not explicitly stipulate that the classifier is also “configured to classify the concept terms” or “configured to classify the resources”
  • on the contrary, the description on page 6, lines 10 to 13 discloses that it is the concept term scoring system and not the classifier that orders the concept terms based on the alternative representation
  • moreover, not all image classifications solve a technical problem
  • if a user classifies displayed images via a user interface to have the images arranged according to the user’s viewing preferences, the image classification will, in most cases, not solve a technical problem
  • in the absence of any technical effect beyond its mere implementation in one or more computers, the subject matter of claim 1 of the first to third auxiliary requests cannot be considered to involve an inventive step

Classification/BDGB ENTERPRISE SOFTWARE – T 1358/09 – 21 November 2014

This decision concerns the patentability of the classification of text documents. In this context, the Board clarifies whether the determination of claim features contributing to the technical character is made without reference to the prior art.

Object of the Invention:

  • the invention is concerned with the computerized classification of text documents
  • this is done by first building a “classification model” and then classifying documents using this classification model

Board I (sufficiency of disclosure):

  • the application does not explain several techniques in detail, and claim 1 does not specify any measure being taken to ensure linear separability
  • it may therefore be questioned whether the application is sufficiently disclosed over the whole scope claimed
  • however, the Board considers that this issue does not prevent it from examining for the presence of an inventive step
  • given the outcome of this examination the question of sufficiency of disclosure need not be answered

Board II (inventive step):

  • claim 1 defines a method for classifying text documents essentially in terms of an abstract mathematical algorithm
  • a mathematical algorithm contributes to the technical character of a computer-implemented method only in so far as it serves a technical purpose (T 1784/06)
  • in the present case, the algorithm serves the general purpose of classifying text documents
  • classification of text documents is certainly useful, as it may help to locate text documents with a relevant cognitive content, but does not qualify as a technical purpose
  • whether two text documents in respect of their textual content belong to the same “class” of documents is not a technical issue
  • the same position was taken in T 1316/09 which held that methods of text classification per se did not produce a relevant technical effect or provide a technical solution to any technical problem

Appellant I (inventive step):

  • the claimed invention could not be seen as the straightforward implementation of something which had been done manually before
  • when manually classifying a text document, a human being would read it through and assign a particular class to it on the basis of his understanding of the document
  • as was known from the domain of cognitive psychology, he would not consider all of the words in the document; words near its beginning would often already provide a clear indication of its semantic topic
  • the claimed automatic classification method on the other hand involved precise computation steps which no human being would ever perform when classifying documents
  • the claimed computerised method was highly efficient, in particular in comparison to classification methods disclosed in documents cited in the international search report

Board III (inventive step):

  • the Board agrees that a human being would not apply the claimed classification method to perform the task of classifying text documents
  • the Board accepts that the proposed computerised method may be faster than classification methods known from the prior art
  • however, the determination of the claim features which contribute to the technical character of the invention is made, at least in principle, without reference to the prior art (T 154/04)
  • it follows that a comparison with what a human being would do or with what is known from the prior art is not a suitable basis for distinguishing between technical and non-technical steps (T 1954/08)

Board IV (inventive step):

  • nevertheless, not all efficiency aspects of an algorithm are by definition without relevance for the question of whether the algorithm provides a technical contribution
  • if an algorithm is particularly suitable for being performed on a computer in that its design was motivated by technical considerations of the internal functioning of the computer, it may arguably be considered to provide a technical contribution to the invention (T 258/03)
  • however, such technical considerations must go beyond merely finding a computer algorithm to carry out some procedure (G 3/08)
  • in the present case no such technical considerations are present
  • the algorithm underlying the method of claim 1 does not go beyond a particular mathematical formulation of the task of classifying documents
  • the aim of this formulation is clearly to enable a computer to carry out this task, but no further consideration of the internal functioning of a computer can be recognised

Appellant II (inventive step):

  • the claimed method provided more reliable and objective results than manual classification, since it was independent of the human subjective understanding of the content of the documents

Board V (inventive step):

  • the Board does not contest that the claimed classification method may provide reliable and objective results, but this is an inherent property of deterministic algorithms
  • the mere fact that an algorithm leads to reproducible results does not imply that it makes a technical contribution
  • since the mathematical algorithm does not contribute to the technical character of the claimed method, an inventive step can be present only in its technical implementation
  • the technical implementation of the mathematical algorithm being obvious
  • –> no inventive step

Information from the author: “technical purpose” is no longer sufficient according to decision G1/19 for non-technical features to make a technical contribution. According to current case law (as of August 2024), in such a case a further or intended or implied technical use is required.

Equivalent Aortic Pressure/ARC SEIBERSDORF – T 0161/18 – 12 May 2020

This decision concerns an invention involving machine learning. The Board of Appeal ruled on the need to disclose training data in a patent application.

Object of the Invention:

  • use of an artificial neural network to transform the blood pressure curve measured at the periphery into the equivalent aortic pressure
  • Claim 1 differs from the closest prior art (general purpose computer) in that the transformation of the blood pressure curve measured at the periphery into the equivalent aortic pressure is carried out with the aid of an artificial neural network whose weighting values are determined by learning’

Board I (sufficiency of disclosure (Article 83 EPC)):

  • the application does not disclose which input data are suitable for training the artificial neural network according to the invention, or at least one data set suitable for solving the present technical problem
  • the training of the artificial neural network can therefore not be reworked by the person skilled in the art and the person skilled in the art can therefore not carry out the invention
  • no sufficient disclosure, since the training according to the invention cannot be carried out due to a lack of corresponding disclosure

Appellant (inventive step):

  • the use of an artificial neural network has the technical effect that the cardiac output can be determined reliably and precisely, while keeping the computing effort within reasonable limits, which enables integration into a mobile and correspondingly handy device

Board II (inventive step):

  • neither the claim nor the description contain details regarding the training of the neural network
  • the claimed neural network is therefore not adapted for the specific claimed application
  • the claimed effect is not achieved in the claimed method over the entire claimed range
  • the effect cannot therefore be considered as an improvement over the prior art when assessing the inventive step
  • the object is to provide an alternative to the method disclosed in D1
  • the use of a neural networks not only corresponds to a general trend in technology, it was also already known for the transformation of the blood pressure curve measured at the periphery into the equivalent aortic pressure
  • the subject-matter of claim 1 was therefore suggested to the skilled person by combining the teaching of D1 with his general technical knowledge or with the teaching of D8 –> no inventive step

Conclusion

  • in this decision, the distinguishing feature is based on machine learning (ML)
  • ML is data-driven and, therefore, the success of an ML invention will largely depend on the data on which it is trained
  • if there is too little suitable training data, it may not work
  • the application does not disclose any input data or data set
  • for AI patent applications, at least some effort should be made to explain:
    • what training data is used, and
    • why enough of it is available to train the ML system appropriately

Classifying resources using a deep network/GOOGLE – T 0874/19 – 6 July, 2022

In this decision, the EPO’s Board of Appeal clarifies that the (previously important) decision T 1227/05 is no longer followed due to the decision of the Enlarged Board of Appeal G1/19.

Object of the Invention

  • classifying search engine resources (e.g. a blog) as a spam resource or not a spam resource
  • subject matter of the invention differs from the closest prior art in that a classifier is configured to generate a respective category score for e.g. a “spam” and “not spam” category, wherein each of the respective category scores measure a predicted likelihood that a resource is a “spam” resource

Appellant:

  • following decision T 1227/05, the claimed system could at least be regarded as simulating a hardware circuit that classifies inputs and thus had a technical purpose

Board:

  • decision T 1227/05 cannot be followed as argued by the appellant in view of recent decision G 1/19, Reasons 133
  • absence of any technical effect beyond its straightforward implementation in one or more computers, the subject-matter does not involve an inventive step

Sparsely connected neural network/MITSUBISHI – T 702/20 – 7 November 2022

In this decision, the EPO Board of Appeal provides a comprehensive technical and legal overview of a neural network “as such”.

Object of the Invention:

  • the application relates to a neural network (NN) apparatus
  • claim 1 differs from the closest prior art in that the different layers of a NN are connected in accordance with an error code check matrix

Examination Division:

  • distinguishing features do not serve a technical purpose and do not relate to a specific technical implementation
  • distinguishing features pertain to initial, fixed structural definition of an abstract mathematical neural network-like model with unknown input and output data

Appellant (part I):

  • claimed NN apparatus has a new and non-obvious structure
  • claimed NN solved a technical problem by providing effects within the computer related to the implementation of NN (storage requirements)

Board (part I):

  • the network structure only defines a class of mathematical functions which, as such, is excluded matter
  • while the storage and computational requirements are reduced in comparison with the fully-connected NN, this does not translate to a technical effect, for the simple reason that the modified NN is different and will not learn in the same way
  • the NN requires less storage, but it does not do the same thing
  • for instance, a one neuron NN requires the least storage, but it will not be able to learn any complex data relationship

Appellant (part II):

  • NN generally solve technical problems by automating human tasks
  • NN apparatuses are artificial brains and that artificial brains solve an automation problem, because they can carry out various complex tasks, instead of the human, without being programmed specifically for one task or another

Board (part II):

  • no evidence that NN functions like a human brain
  • whilst the functioning of NN may not be foreseeable prior to training and the programmer may not understand the significance of its individual parameters, the NN still operates according to the programming of its structure and learning scheme
  • the claims do not further specify any particular task, i.e. type of relationship to be learned, for the NN à NN does not solve any specific automation problem

Appellant (part III):

  • a technical problem may also be solved if the outputs of the system have an implied further technical use (G 1/19, reasons 137)

Board (part III):

  • claimed learning and use of the NN “to solve a classification problem or a regression problem” can use any data
  • outputs of the NN do not have any implied “further technical use”
  • outputs may be related to forecasting stock market evolution

Conclusion:

This decision provides a technical background regarding NN (reasons 7 to 8.2) and a legal background regarding NN (reasons 9 to 11.3), which provides a good insight into this topic.

Furthermore, the below figure shows according to G 1/19, point 85 and 86 how and when “technical effects” or “technical interactions” based on inter alia non-technical features may occur in the context of a computer-implemented process (the arrows in the figure above represent interactions and not abstract data). In this decision T 702/20 it was discussed whether the NN/ non-technical features contribute to the technical character of the invention via the output side, via the input side and via the technical implementation.