Nokia Open-Sources AnyJev LLM Decision Model Library

Nokia's applied research team has open-sourced AnyJev, a Python library designed to transform open large language models (LLMs) into decision models without requiring any additional training. This innovation targets a common production use case: selecting a single, definitive answer from a predefined set of options, rather than generating free-form text. The AnyJev library is readily deployable, installable via PyPI, and distributed under the Apache-2.0 license. It supports both transformers and vLLM backends, leveraging shared-prefix scoring for its operations. The interface of AnyJev is inspired by Jev, a decision model developed by System One and launched by TypeSafe AI in September 2026. With AnyJev, users can input a typed question and receive a decision along with an associated probability. This probability can be thresholded to determine the final output. The probability is derived directly from the model's next-token distribution, eliminating the need for text generation, parsing, or fine-tuning. The library accommodates three primary question types: choice questions, which select one option from a list of K possibilities; noul questions, which are binary yes/no inquiries; and score questions, which categorize an answer into one of several ordered bins. The development of AnyJev addresses significant flaws identified in simpler methods that directly read logits from LLMs. These simpler approaches often restrict the next token to option labels and then read the associated scores. However, the Nokia research team highlights two critical shortcomings of this shortcut. Firstly, the order in which options are presented can alter the final answer, indicating instability. Secondly, the probabilities generated by such methods are frequently uncalibrated. The Nokia team attributes these calibration issues to two primary causes: prior bias, where the model inherently favors certain labels (e.g., 'Yes' over 'No') irrespective of the input context, and position bias, where the model demonstrates a preference for specific positions within the list of options. To counteract these issues, AnyJev implements a two-level approach, referred to as L0 and L1, with L0 being enabled by default. The L0 level incorporates two key fixes. The first is cyclic shifts, a technique applied to questions with K options. For each question, the list of options is rotated K times, ensuring that every option appears in every possible position exactly once. The results from these rotations are then combined in log space using a geometric mean. This method effectively neutralizes position bias, assuming it operates additively in logit space. The second fix is prior correction. By default, AnyJev employs batch calibration, maintaining a running mean of the predicted distributions across real-world inputs. This running mean is then divided out at a strength of 0.75. The prior correction mechanism initiates its process after approximately 8 initial inputs have been processed.