RED-9352: Implemented rules execution co-pilot
This commit is contained in:
Dominique Eifländer
parent
e44969dd7a
commit
da3cd318a3
93
README.md
93
README.md
@ -1,93 +0,0 @@
|
||||
# Llm Service
|
||||
|
||||
|
||||
|
||||
## Getting started
|
||||
|
||||
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
|
||||
|
||||
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
|
||||
|
||||
## Add your files
|
||||
|
||||
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
|
||||
- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
|
||||
|
||||
```
|
||||
cd existing_repo
|
||||
git remote add origin https://gitlab.knecon.com/fforesight/llm-service.git
|
||||
git branch -M main
|
||||
git push -uf origin main
|
||||
```
|
||||
|
||||
## Integrate with your tools
|
||||
|
||||
- [ ] [Set up project integrations](https://gitlab.knecon.com/fforesight/llm-service/-/settings/integrations)
|
||||
|
||||
## Collaborate with your team
|
||||
|
||||
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
|
||||
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
|
||||
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
|
||||
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
|
||||
- [ ] [Set auto-merge](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
|
||||
|
||||
## Test and Deploy
|
||||
|
||||
Use the built-in continuous integration in GitLab.
|
||||
|
||||
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
|
||||
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
|
||||
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
|
||||
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
|
||||
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
|
||||
|
||||
***
|
||||
|
||||
# Editing this README
|
||||
|
||||
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
|
||||
|
||||
## Suggestions for a good README
|
||||
|
||||
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
|
||||
|
||||
## Name
|
||||
Choose a self-explaining name for your project.
|
||||
|
||||
## Description
|
||||
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
|
||||
|
||||
## Badges
|
||||
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
|
||||
|
||||
## Visuals
|
||||
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
|
||||
|
||||
## Installation
|
||||
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
|
||||
|
||||
## Usage
|
||||
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
|
||||
|
||||
## Support
|
||||
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
|
||||
|
||||
## Roadmap
|
||||
If you have ideas for releases in the future, it is a good idea to list them in the README.
|
||||
|
||||
## Contributing
|
||||
State if you are open to contributions and what your requirements are for accepting them.
|
||||
|
||||
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
|
||||
|
||||
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
|
||||
|
||||
## Authors and acknowledgment
|
||||
Show your appreciation to those who have contributed to the project.
|
||||
|
||||
## License
|
||||
For open source projects, say how it is licensed.
|
||||
|
||||
## Project status
|
||||
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
|
||||
149
build.gradle.kts
Normal file
149
build.gradle.kts
Normal file
@ -0,0 +1,149 @@
|
||||
import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
|
||||
|
||||
plugins {
|
||||
java
|
||||
id("org.springframework.boot") version "3.1.5"
|
||||
id("io.spring.dependency-management") version "1.1.4"
|
||||
id("org.sonarqube") version "4.4.1.3373"
|
||||
id("io.freefair.lombok") version "8.6"
|
||||
pmd
|
||||
checkstyle
|
||||
jacoco
|
||||
}
|
||||
|
||||
group = "com.knecon.fforesight"
|
||||
java.sourceCompatibility = JavaVersion.VERSION_17
|
||||
|
||||
configurations {
|
||||
compileOnly {
|
||||
extendsFrom(configurations.annotationProcessor.get())
|
||||
}
|
||||
}
|
||||
|
||||
pmd {
|
||||
isConsoleOutput = true
|
||||
}
|
||||
|
||||
tasks.pmdMain {
|
||||
pmd.ruleSetFiles = files("${projectDir}/config/pmd/pmd.xml")
|
||||
}
|
||||
|
||||
tasks.pmdTest {
|
||||
pmd.ruleSetFiles = files("${projectDir}/config/pmd/test_pmd.xml")
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
tasks.jacocoTestReport {
|
||||
reports {
|
||||
xml.required.set(false)
|
||||
csv.required.set(false)
|
||||
html.outputLocation.set(layout.buildDirectory.dir("jacocoHtml"))
|
||||
}
|
||||
}
|
||||
|
||||
repositories {
|
||||
mavenLocal()
|
||||
mavenCentral()
|
||||
maven {
|
||||
url = uri("https://nexus.knecon.com/repository/gindev/");
|
||||
credentials {
|
||||
username = providers.gradleProperty("mavenUser").getOrNull();
|
||||
password = providers.gradleProperty("mavenPassword").getOrNull();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tasks.register("publish") {
|
||||
|
||||
}
|
||||
|
||||
tasks.named<BootBuildImage>("bootBuildImage") {
|
||||
|
||||
|
||||
environment.put("BPE_DELIM_JAVA_TOOL_OPTIONS", " ")
|
||||
environment.put("BPE_APPEND_JAVA_TOOL_OPTIONS", "-Dfile.encoding=UTF-8")
|
||||
|
||||
imageName.set("nexus.knecon.com:5001/ff/${project.name}:${project.version}")
|
||||
if (project.hasProperty("buildbootDockerHostNetwork")) {
|
||||
network.set("host")
|
||||
}
|
||||
docker {
|
||||
if (project.hasProperty("buildbootDockerHostNetwork")) {
|
||||
bindHostToBuilder.set(true)
|
||||
}
|
||||
verboseLogging.set(true)
|
||||
|
||||
publishRegistry {
|
||||
username.set(providers.gradleProperty("mavenUser").getOrNull())
|
||||
password.set(providers.gradleProperty("mavenPassword").getOrNull())
|
||||
email.set(providers.gradleProperty("mavenEmail").getOrNull())
|
||||
url.set("https://nexus.knecon.com:5001/")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
configurations {
|
||||
all {
|
||||
exclude(group = "org.springframework.boot", module = "spring-boot-starter-logging")
|
||||
exclude(group = "commons-logging", module = "commons-logging")
|
||||
}
|
||||
}
|
||||
|
||||
extra["springCloudVersion"] = "2022.0.5"
|
||||
extra["testcontainersVersion"] = "1.19.7"
|
||||
|
||||
dependencies {
|
||||
implementation("org.springframework.boot:spring-boot-starter-actuator")
|
||||
implementation("org.springframework.boot:spring-boot-starter-amqp")
|
||||
implementation("org.springframework.boot:spring-boot-starter-web")
|
||||
implementation("org.springframework.cloud:spring-cloud-starter-openfeign")
|
||||
implementation("org.springframework.boot:spring-boot-starter-websocket")
|
||||
implementation("org.springframework.security:spring-security-messaging:6.1.3")
|
||||
implementation("com.iqser.red.commons:storage-commons:2.49.0")
|
||||
implementation("com.knecon.fforesight:keycloak-commons:0.29.0")
|
||||
implementation("com.knecon.fforesight:swagger-commons:0.7.0")
|
||||
implementation("com.azure:azure-ai-openai:1.0.0-beta.5")
|
||||
developmentOnly("org.springframework.boot:spring-boot-devtools")
|
||||
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
|
||||
testImplementation("org.springframework.boot:spring-boot-starter-test")
|
||||
testImplementation("org.springframework.amqp:spring-rabbit-test")
|
||||
implementation("ch.qos.logback:logback-classic")
|
||||
}
|
||||
|
||||
|
||||
|
||||
dependencyManagement {
|
||||
imports {
|
||||
mavenBom("org.testcontainers:testcontainers-bom:${property("testcontainersVersion")}")
|
||||
mavenBom("org.springframework.cloud:spring-cloud-dependencies:${property("springCloudVersion")}")
|
||||
}
|
||||
}
|
||||
|
||||
tasks.withType<Test> {
|
||||
minHeapSize = "1024m"
|
||||
maxHeapSize = "2048m"
|
||||
useJUnitPlatform()
|
||||
reports {
|
||||
junitXml.outputLocation.set(layout.buildDirectory.dir("reports/junit"))
|
||||
}
|
||||
}
|
||||
|
||||
sonarqube {
|
||||
properties {
|
||||
providers.gradleProperty("sonarToken").getOrNull()?.let { property("sonar.login", it) }
|
||||
property("sonar.host.url", "https://sonarqube.knecon.com")
|
||||
}
|
||||
}
|
||||
|
||||
tasks.test {
|
||||
finalizedBy(tasks.jacocoTestReport) // report is always generated after tests run
|
||||
}
|
||||
tasks.jacocoTestReport {
|
||||
dependsOn(tasks.test) // tests are required to run before generating the report
|
||||
reports {
|
||||
xml.required.set(true)
|
||||
csv.required.set(false)
|
||||
}
|
||||
}
|
||||
38
config/checkstyle/checkstyle.xml
Normal file
38
config/checkstyle/checkstyle.xml
Normal file
@ -0,0 +1,38 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE module PUBLIC "-//Puppy Crawl//DTD Check Configuration 1.3//EN"
|
||||
"http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
|
||||
<module name="Checker">
|
||||
<property
|
||||
name="severity"
|
||||
value="error"/>
|
||||
<module name="TreeWalker">
|
||||
<module name="SuppressWarningsHolder"/>
|
||||
<module name="MissingDeprecated"/>
|
||||
<module name="MissingOverride"/>
|
||||
<module name="AnnotationLocation"/>
|
||||
<module name="JavadocStyle"/>
|
||||
<module name="NonEmptyAtclauseDescription"/>
|
||||
<module name="IllegalImport"/>
|
||||
<module name="RedundantImport"/>
|
||||
<module name="RedundantModifier"/>
|
||||
<module name="EmptyBlock"/>
|
||||
<module name="DefaultComesLast"/>
|
||||
<module name="EmptyStatement"/>
|
||||
<module name="EqualsHashCode"/>
|
||||
<module name="IllegalInstantiation"/>
|
||||
<module name="ModifiedControlVariable"/>
|
||||
<module name="MultipleVariableDeclarations"/>
|
||||
<module name="PackageDeclaration"/>
|
||||
<module name="ParameterAssignment"/>
|
||||
<module name="SimplifyBooleanExpression"/>
|
||||
<module name="SimplifyBooleanReturn"/>
|
||||
<module name="StringLiteralEquality"/>
|
||||
<module name="OneStatementPerLine"/>
|
||||
<module name="FinalClass"/>
|
||||
<module name="ArrayTypeStyle"/>
|
||||
<module name="UpperEll"/>
|
||||
<module name="OuterTypeFilename"/>
|
||||
</module>
|
||||
<module name="FileTabCharacter"/>
|
||||
<module name="SuppressWarningsFilter"/>
|
||||
</module>
|
||||
16
config/pmd/pmd.xml
Normal file
16
config/pmd/pmd.xml
Normal file
@ -0,0 +1,16 @@
|
||||
<?xml version="1.0"?>
|
||||
<ruleset name="Custom Rules"
|
||||
xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
|
||||
|
||||
<description>Knecon test pmd rules</description>
|
||||
|
||||
<rule ref="category/java/errorprone.xml">
|
||||
<exclude name="DataflowAnomalyAnalysis"/>
|
||||
<exclude name="MissingSerialVersionUID"/>
|
||||
<exclude name="AvoidLiteralsInIfCondition"/>
|
||||
<exclude name="BeanMembersShouldSerialize"/>
|
||||
<exclude name="AvoidDuplicateLiterals"/>
|
||||
</rule>
|
||||
</ruleset>
|
||||
22
config/pmd/test_pmd.xml
Normal file
22
config/pmd/test_pmd.xml
Normal file
@ -0,0 +1,22 @@
|
||||
<?xml version="1.0"?>
|
||||
<ruleset name="Custom ruleset"
|
||||
xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
|
||||
|
||||
<description>
|
||||
Knecon test ruleset checks the code for bad stuff
|
||||
</description>
|
||||
|
||||
|
||||
<rule ref="category/java/errorprone.xml">
|
||||
<exclude name="MissingSerialVersionUID"/>
|
||||
<exclude name="AvoidLiteralsInIfCondition"/>
|
||||
<exclude name="AvoidDuplicateLiterals"/>
|
||||
<exclude name="NullAssignment"/>
|
||||
<exclude name="AssignmentInOperand"/>
|
||||
<exclude name="TestClassWithoutTestCases"/>
|
||||
<exclude name="BeanMembersShouldSerialize"/>
|
||||
</rule>
|
||||
|
||||
</ruleset>
|
||||
1
gradle.properties.kts
Normal file
1
gradle.properties.kts
Normal file
@ -0,0 +1 @@
|
||||
version = 1.0-SNAPSHOT
|
||||
8
publish-custom-image.sh
Executable file
8
publish-custom-image.sh
Executable file
@ -0,0 +1,8 @@
|
||||
#!/bin/bash
|
||||
dir=${PWD##*/}
|
||||
gradle assemble
|
||||
|
||||
buildNumber=${1:-1}
|
||||
|
||||
gradle bootBuildImage --cleanCache --publishImage -PbuildbootDockerHostNetwork=true -Pversion=$USER-$buildNumber
|
||||
echo "nexus.knecon.com:5001/red/${dir}-server-v1:$USER-$buildNumber"
|
||||
6
renovate.json
Normal file
6
renovate.json
Normal file
@ -0,0 +1,6 @@
|
||||
{
|
||||
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
|
||||
"extends": [
|
||||
"config:base"
|
||||
]
|
||||
}
|
||||
1
settings.gradle.kts
Normal file
1
settings.gradle.kts
Normal file
@ -0,0 +1 @@
|
||||
rootProject.name = "llm-service"
|
||||
@ -0,0 +1,30 @@
|
||||
package com.knecon.fforesight.llm.service;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.scheduling.annotation.EnableAsync;
|
||||
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
|
||||
|
||||
import com.iqser.red.storage.commons.StorageAutoConfiguration;
|
||||
import com.knecon.fforesight.keycloakcommons.DefaultKeyCloakCommonsAutoConfiguration;
|
||||
import com.knecon.fforesight.swaggercommons.SpringDocAutoConfiguration;
|
||||
import com.knecon.fforesight.tenantcommons.MultiTenancyAutoConfiguration;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@EnableWebMvc
|
||||
@EnableAsync
|
||||
@Import({StorageAutoConfiguration.class})
|
||||
@ImportAutoConfiguration({StorageAutoConfiguration.class, MultiTenancyAutoConfiguration.class, SpringDocAutoConfiguration.class, DefaultKeyCloakCommonsAutoConfiguration.class})
|
||||
@SpringBootApplication
|
||||
public class Application {
|
||||
|
||||
public static void main(String[] args) {
|
||||
|
||||
SpringApplication.run(Application.class, args);
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
package com.knecon.fforesight.llm.service.api;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
@Data
|
||||
@RequiredArgsConstructor
|
||||
public class ErrorMessage {
|
||||
|
||||
private final String message;
|
||||
private OffsetDateTime timestamp = OffsetDateTime.now();
|
||||
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
package com.knecon.fforesight.llm.service.api.model;
|
||||
|
||||
import lombok.AllArgsConstructor;
|
||||
import lombok.Builder;
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public class ChatEvent {
|
||||
|
||||
String token;
|
||||
|
||||
}
|
||||
@ -0,0 +1,19 @@
|
||||
package com.knecon.fforesight.llm.service.api.model;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import lombok.AllArgsConstructor;
|
||||
import lombok.Builder;
|
||||
import lombok.Data;
|
||||
import lombok.NoArgsConstructor;
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public class PromptList {
|
||||
|
||||
private List<String> prompts = new ArrayList<>();
|
||||
|
||||
}
|
||||
@ -0,0 +1,19 @@
|
||||
package com.knecon.fforesight.llm.service.controller.advice;
|
||||
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.ExceptionHandler;
|
||||
import org.springframework.web.bind.annotation.RestControllerAdvice;
|
||||
import org.springframework.web.server.ResponseStatusException;
|
||||
|
||||
import com.knecon.fforesight.llm.service.api.ErrorMessage;
|
||||
|
||||
@RestControllerAdvice
|
||||
public class ControllerAdvice {
|
||||
|
||||
@ExceptionHandler(ResponseStatusException.class)
|
||||
public ResponseEntity<ErrorMessage> handleResponseStatusException(ResponseStatusException e) {
|
||||
|
||||
return new ResponseEntity<>(new ErrorMessage(e.getMessage()), e.getStatusCode());
|
||||
}
|
||||
|
||||
}
|
||||
43
src/main/java/com/knecon/fforesight/llm/service/controller/external/LlmContoller.java
vendored
Normal file
43
src/main/java/com/knecon/fforesight/llm/service/controller/external/LlmContoller.java
vendored
Normal file
@ -0,0 +1,43 @@
|
||||
package com.knecon.fforesight.llm.service.controller.external;
|
||||
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.ResponseBody;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
import com.knecon.fforesight.keycloakcommons.security.KeycloakSecurity;
|
||||
import com.knecon.fforesight.llm.service.api.model.PromptList;
|
||||
import com.knecon.fforesight.llm.service.services.LlmService;
|
||||
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
import io.swagger.v3.oas.annotations.responses.ApiResponse;
|
||||
import io.swagger.v3.oas.annotations.responses.ApiResponses;
|
||||
import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
import io.swagger.v3.oas.annotations.tags.Tags;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@RestController
|
||||
@RequiredArgsConstructor
|
||||
@RequestMapping("${fforesight.llm-service.base-path:/api/llm}")
|
||||
@Tags(value = {@Tag(name = "LLM", description = "Provides endpoint to call llm")})
|
||||
public class LlmContoller {
|
||||
|
||||
private final LlmService llmService;
|
||||
|
||||
|
||||
@ResponseBody
|
||||
// @PreAuthorize("hasAuthority('" + LLM + "')")
|
||||
@Operation(summary = "Make a request to the llm, response will be streamed to websocket.")
|
||||
@ApiResponses(value = {@ApiResponse(responseCode = "200"), @ApiResponse(responseCode = "400", description = "Bad Request. Something is not right in the request object. See response for details."), @ApiResponse(responseCode = "401", description = "Unauthorized."), @ApiResponse(responseCode = "403", description = "Forbidden.")})
|
||||
@PostMapping(value = "/chat-async", consumes = MediaType.APPLICATION_JSON_VALUE)
|
||||
public void chat(@RequestBody PromptList promptList) {
|
||||
|
||||
log.info("UserId is: {}", KeycloakSecurity.getUserId());
|
||||
llmService.execute(promptList.getPrompts());
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,325 @@
|
||||
package com.knecon.fforesight.llm.service.model;
|
||||
|
||||
public class SystemMessages {
|
||||
|
||||
public static String RULES_CO_PILOT = """
|
||||
From now on, you are a Drools rule generator. This means you will start your answer with a step-by-step explanation how to write a rule, which will fulfill the prompt, followed by the rule.
|
||||
|
||||
You have a document structure written in Java with the following objects:
|
||||
|
||||
- Section
|
||||
- Table
|
||||
- TableCell
|
||||
- Paragraph
|
||||
- Headline
|
||||
- Page
|
||||
- TextEntity
|
||||
- EntityCreationService
|
||||
|
||||
The Section, Table, TableCell, Paragraph, and Headline implement a common interface called SemanticNode. SemanticNodes are arranged in a tree-like fashion, where any SemanticNode can have multiple SemanticNodes as children. The arrangement is as follows:
|
||||
- Tables only have TableCells as children.
|
||||
- TableCells may have any child, except TableCells.
|
||||
- Paragraphs and Headlines have no children.
|
||||
- Sections may have any child except TableCells, but if it contains Paragraphs as well as Tables, it is split into a Section with multiple Sections as children, where any child Section only contains either Tables or Paragraphs.
|
||||
Further, if the first SemanticNode is a Headline it remains the first child in the Parent Section, before any subsections. You can assume there are no null values.
|
||||
It is also important to assume, that the document structure might be faulty in its design and to write the rule as robust as possible.
|
||||
|
||||
The goal of the rules is to identify pieces of text that should be extracted. In order to represent the pieces of text we create a TextEntity using the entityCreationService.
|
||||
For example, we want to extract all Authors and addresses of a document. Or all personally identifiable information, such as E-Mails and Telephone Numbers.
|
||||
TextEntities may also represent other pieces of text, such as published information, or certain species of vertebrates.
|
||||
The TextEntities are part of the document structure, such that they are referenced in each SemanticNode and Page which contains it. Further, the TextEntity references each Page and SemanticNode whose text it intersects.
|
||||
|
||||
A rule is written by first enforcing conditions on a SemanticNode in the when-block and then a call to the EntityCreationService is performed in the then-block.
|
||||
The created entity then needs to be applied or removed, depending on the use case. Apply means it will be extracted, removed means it will not.
|
||||
It is important to only call methods that change the state inside the then-block and NEVER call them in the when block. Not even with the "from" keyword.
|
||||
This includes all entityCreationService methods or apply/remove on a TextEntity,
|
||||
Finally, the rule has to be formatted as such: the name starts with an Identifier following the pattern /w+./d+./d+, then a ":" and finally a descriptive name.
|
||||
The identifier must match with the identifier used in the apply/remove calls on the TextEntity.
|
||||
|
||||
Examples:
|
||||
A rule which finds all text until the end of a line, when the line starts with "Authors:" as the type "CBI_author".
|
||||
This rule searches for any section containing the String "Authors:" and then creates a TextEntity after that string in that section.
|
||||
rule "CBI.17.0: Add CBI_author for lines after AUTHORS:"
|
||||
when
|
||||
$section: Section(containsString("Authors:"))
|
||||
then
|
||||
entityCreationService.lineAfterString("Authors:", "CBI_author", EntityType.ENTITY, $section)
|
||||
.forEach(entity -> entity.apply("CBI.17.0", "Line after \\"Authors:\\""));
|
||||
end
|
||||
|
||||
A rule, which finds contact information as type "PII" by looking at the text after certain keywords:
|
||||
This rule iterates through all relevant keywords and sections, for each match the lineAfterString method is called, creating a new TextEntity.
|
||||
rule "PII.4.0: Extract line after contact information keywords"
|
||||
when
|
||||
$contactKeyword: String() from List.of("Contact point:",
|
||||
"Contact:",
|
||||
"Alternative contact:",
|
||||
"European contact:",
|
||||
"No:",
|
||||
"Contact:",
|
||||
"Tel.:",
|
||||
"Tel:",
|
||||
"Telephone number:",
|
||||
"Telephone No:",
|
||||
"Telephone:",
|
||||
"Phone No.",
|
||||
"Phone:",
|
||||
"Fax number:",
|
||||
"Fax:",
|
||||
"E-mail:",
|
||||
"Email:",
|
||||
"e-mail:",
|
||||
"E-mail address:")
|
||||
$section: Section(containsString($contactKeyword))
|
||||
then
|
||||
entityCreationService.lineAfterString($contactKeyword, "PII", EntityType.ENTITY, $section)
|
||||
.forEach(contactEntity -> contactEntity.apply("PII.4.0", "Found after \\"" + $contactKeyword + "\\" contact keyword");
|
||||
end
|
||||
|
||||
A rule which finds all Emails as type "PII" using a regex:
|
||||
This rule first searches for a section containing the character "@", since the execution of the regex might be costly to run on all sections.
|
||||
rule "PII.1.0: Extract Emails by RegEx"
|
||||
when
|
||||
$section: Section(containsString("@"))
|
||||
then
|
||||
entityCreationService.byRegex("\\\\b([A-Za-z0-9._%+\\\\-]+@[A-Za-z0-9.\\\\-]+\\\\.[A-Za-z\\\\-]{1,23}[A-Za-z])\\\\b", "PII", EntityType.ENTITY, 1, $section)
|
||||
.forEach(emailEntity -> emailEntity.apply("PII.1.0", "Found by Email Regex");
|
||||
end
|
||||
|
||||
A rule which extracts all paragraphs of a section with the Headline "Mortality"
|
||||
This rule first identifies a Headline containing the word "Mortality" and then extracts all paragraphs as TextEntities using the entityCreationService.
|
||||
rule "DOC.14.0: Mortality"
|
||||
when
|
||||
$headline: Headline(containsString("Mortality"))
|
||||
then
|
||||
entityCreationService.bySemanticNodeParagraphsOnly($headline.getParent(), "mortality", EntityType.ENTITY)
|
||||
.forEach(entity -> entity.apply("DOC.14.0", "Mortality found", "n-a"));
|
||||
end
|
||||
|
||||
A rule which extracts all paragraphs of a Section and its Subsections with the Headline "Study Design"
|
||||
This rule uses the SectionIdentifier to find all headlines which are children of the headline "Study Design". From these headlines it finds the sections using the getParent() method.
|
||||
This is done, since the document structure might be faulty, so using the SectionIdentifier is more robust. The SectionIdentifier describes the numbers in front of a Headline,
|
||||
e.g. "3.0 Study Design" -> "3.0"
|
||||
rule "DOC.20.1: Study Design"
|
||||
when
|
||||
Headline(containsStringIgnoreCase("Study Design"), $sectionIdentifier: getSectionIdentifier())
|
||||
$headline: Headline(getSectionIdentifier().isChildOf($sectionIdentifier))
|
||||
then
|
||||
entityCreationService.bySemanticNodeParagraphsOnly($headline.getParent(), "study_design", EntityType.ENTITY)
|
||||
.forEach(entity -> {
|
||||
entity.apply("DOC.20.1", "Study design section found", "n-a");
|
||||
});
|
||||
end
|
||||
|
||||
A rule which extracts each TableCell with the header 'Author' or 'Author(s)'
|
||||
This rule first identifies a header TableCell containing the word 'Author' or 'Author(s)' and then finds any TableCell underneath it and creates a TextEntity from it.
|
||||
rule "CBI.11.0: Extract and recommend TableCell with header 'Author' or 'Author(s)'"
|
||||
when
|
||||
$table: Table(hasHeader("Author(s)") || hasHeader("Author"))
|
||||
TableCell(isHeader(), containsAnyStringIgnoreCase("Author", "Author(s)"), $authorCol: col) from $table.streamHeaders().toList()
|
||||
$authorCell: TableCell() from $table.streamCol($authorCol).toList()
|
||||
then
|
||||
entityCreationService.bySemanticNode($authorCell, "CBI_author", EntityType.ENTITY)
|
||||
.ifPresent(authorEntity -> authorEntity.apply("CBI.11.0", "Author header found");
|
||||
end
|
||||
|
||||
A rule which extracts all authors in each row that represents a vertebrate study
|
||||
This rule uses the when block to first identify TableCells that are headers containing a String "Author" or "Author(s)" and a header cell "Vertebrate Study Y/N".
|
||||
It then identifies cells underneath the second header containing a String like "Y" or "Yes" and finally a cell in the same row as the previous cell and same column as the Author header cell.
|
||||
Each match is then turned into an entity using the entityCreationService in the then-block.
|
||||
rule "CBI.12.0: Extract and recommend TableCell with header 'Author' or 'Author(s)' and header 'Vertebrate study Y/N' with value 'Yes'"
|
||||
when
|
||||
$table: Table(hasHeader("Author(s)") || hasHeader("Author"), hasHeaderIgnoreCase("Vertebrate Study Y/N"))
|
||||
TableCell(isHeader(), containsAnyStringIgnoreCase("Author", "Author(s)"), $authorCol: col) from $table.streamHeaders().toList()
|
||||
TableCell(isHeader(), containsStringIgnoreCase("Vertebrate study Y/N"), $vertebrateCol: col) from $table.streamHeaders().toList()
|
||||
TableCell(!isHeader(), containsAnyString("Yes", "Y"), $rowWithYes: row) from $table.streamCol($vertebrateCol).toList()
|
||||
$authorCell: TableCell(row == $rowWithYes) from $table.streamCol($authorCol).toList()
|
||||
then
|
||||
entityCreationService.bySemanticNode($authorCell, "CBI_author", EntityType.ENTITY)
|
||||
.ifPresent(authorEntity -> authorEntity.apply("CBI.12.0", "Extracted because it's row belongs to a vertebrate study");
|
||||
end
|
||||
|
||||
|
||||
Below are all functions you can use, listed by their respective class. Make sure to only use functions that are listed here or are included in java 17
|
||||
EntityCreationService:
|
||||
betweenTextRanges(List<TextRange> startBoundaries, List<TextRange> stopBoundaries, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenTextRanges(List<TextRange> startBoundaries, List<TextRange> stopBoundaries, String type, EntityType entityType, SemanticNode node, int limit) -> Stream<TextEntity>
|
||||
byTextRange(TextRange textRange, String type, EntityType entityType, SemanticNode node) -> Optional<TextEntity>
|
||||
byRegexWithLineBreaks(String regexPattern, String type, EntityType entityType, int group, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegexWithLineBreaks(String regexPattern, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegexWithLineBreaksIgnoreCase(String regexPattern, String type, EntityType entityType, int group, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegexWithLineBreaksIgnoreCase(String regexPattern, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegex(String regexPattern, String type, EntityType entityType, int group, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegex(String regexPattern, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegexIgnoreCase(String regexPattern, String type, EntityType entityType, int group, SemanticNode node) -> Stream<TextEntity>
|
||||
byRegexIgnoreCase(String regexPattern, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
bySemanticNode(SemanticNode node, String type, EntityType entityType) -> Optional<TextEntity>
|
||||
betweenStrings(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIgnoreCase(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeStart(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeStartIgnoreCase(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeEnd(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeEndIgnoreCase(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeStartAndEnd(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenStringsIncludeStartAndEndIgnoreCase(String start, String stop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
shortestBetweenAnyString(List<String> starts, List<String> stops, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
shortestBetweenAnyStringIgnoreCase(List<String> starts, List<String> stops, String type, EntityType entityType, SemanticNode node, int limit) -> Stream<TextEntity>
|
||||
shortestBetweenAnyStringIgnoreCase(List<String> starts, List<String> stops, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenRegexes(String regexStart, String regexStop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
betweenRegexesIgnoreCase(String regexStart, String regexStop, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
lineAfterStrings(List<String> strings, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
lineAfterStringsIgnoreCase(List<String> strings, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
lineAfterString(String string, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
lineAfterStringIgnoreCase(String string, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
lineAfterStringAcrossColumns(String string, String type, EntityType entityType, Table tableNode) -> Stream<TextEntity>
|
||||
lineAfterStringAcrossColumnsIgnoreCase(String string, String type, EntityType entityType, Table tableNode) -> Stream<TextEntity>
|
||||
semanticNodeAfterString(String string, String type, EntityType entityType, SemanticNode node) -> Optional<TextEntity>
|
||||
byString(String keyword, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
byStringIgnoreCase(String keyword, String type, EntityType entityType, SemanticNode node) -> Stream<TextEntity>
|
||||
bySemanticNodeParagraphsOnly(SemanticNode node, String type, EntityType entityType) -> Stream<TextEntity>
|
||||
bySemanticNodeParagraphsOnlyMergeConsecutive(SemanticNode node, String type, EntityType entityType) -> Stream<TextEntity>
|
||||
byPrefixExpansionRegex(TextEntity entity, String regexPattern) -> Optional<TextEntity>
|
||||
bySuffixExpansionRegex(TextEntity entity, String regexPattern) -> Optional<TextEntity>
|
||||
|
||||
SemanticNode:
|
||||
length() -> int
|
||||
getParent() -> SemanticNode
|
||||
getType() -> NodeType
|
||||
isLeaf() -> boolean
|
||||
getTextBlock() -> TextBlock
|
||||
getPages() -> Set<Page>
|
||||
getPages(TextRange textRange) -> Set<Page>
|
||||
getTextRange() -> TextRange
|
||||
getHeadline() -> Headline
|
||||
getSectionIdentifier() -> SectionIdentifier
|
||||
getNextSibling() -> Optional<SemanticNode>
|
||||
getPreviousSibling() -> Optional<SemanticNode>
|
||||
getEntities() -> Set<TextEntity>
|
||||
getBBox() -> Map<Page, Rectangle2D>
|
||||
streamChildren() -> Stream<SemanticNode>
|
||||
getFirstPage() -> Page
|
||||
onPage(int pageNumber) -> boolean
|
||||
hasParent() -> boolean
|
||||
getHighestParent() -> SemanticNode
|
||||
hasEntitiesOfType(String type) -> boolean
|
||||
hasEntitiesOfAnyType(String... types) -> boolean
|
||||
hasEntitiesOfAllTypes(String... types) -> boolean
|
||||
getEntitiesOfType(List<String> types) -> List<TextEntity>
|
||||
getEntitiesOfType(String type) -> List<TextEntity>
|
||||
getEntitiesOfType(String... types) -> List<TextEntity>
|
||||
hasText() -> boolean
|
||||
containsString(String string) -> boolean
|
||||
containsAllStrings(String... strings) -> boolean
|
||||
containsAnyString(String... strings) -> boolean
|
||||
containsAnyString(List<String> strings) -> boolean
|
||||
containsStringIgnoreCase(String string) -> boolean
|
||||
containsAnyStringIgnoreCase(String... strings) -> boolean
|
||||
containsAllStringsIgnoreCase(String... strings) -> boolean
|
||||
containsWord(String word) -> boolean
|
||||
containsWordIgnoreCase(String word) -> boolean
|
||||
containsAnyWord(String... words) -> boolean
|
||||
containsAnyWordIgnoreCase(String... words) -> boolean
|
||||
containsAllWords(String... words) -> boolean
|
||||
containsAllWordsIgnoreCase(String... words) -> boolean
|
||||
matchesRegex(String regexPattern) -> boolean
|
||||
matchesRegexIgnoreCase(String regexPattern) -> boolean
|
||||
streamChildrenOfType(NodeType nodeType) -> Stream<SemanticNode>
|
||||
streamAllSubNodes() -> Stream<SemanticNode>
|
||||
streamAllSubNodesOfType(NodeType nodeType) -> Stream<SemanticNode>
|
||||
containsRectangle(Rectangle2D rectangle2D, Integer pageNumber) -> boolean
|
||||
intersectsRectangle(int x, int y, int w, int h, int pageNumber) -> boolean
|
||||
getSectionIdentifier() -> SectionIdentifier
|
||||
|
||||
Section:
|
||||
hasTables() -> boolean
|
||||
anyHeadlineContainsString(String value) -> boolean
|
||||
anyHeadlineContainsStringIgnoreCase(String value) -> boolean
|
||||
|
||||
SectionIdentifier:
|
||||
isParentOf(SectionIdentifier sectionIdentifier) -> boolean
|
||||
isChildOf(SectionIdentifier sectionIdentifier) -> boolean
|
||||
|
||||
Table:
|
||||
rowContainsStringsIgnoreCase(Integer row, List<String> strings) -> boolean
|
||||
streamRow(int row) -> Stream<TableCell>
|
||||
streamHeaders() -> Stream<TableCell>
|
||||
streamTableCells() -> Stream<TableCell>
|
||||
streamCol(int col) -> Stream<TableCell>
|
||||
streamTableCellsWithHeader(String header) -> Stream<TableCell>
|
||||
getCell(int row, int col) -> TableCell
|
||||
streamTableCellsWhichContainType(String type) -> Stream<TableCell>
|
||||
streamHeadersForCell(int row, int col) -> Stream<TableCell>
|
||||
hasHeader(String header) -> boolean
|
||||
hasHeaderIgnoreCase(String header) -> boolean
|
||||
hasRowWithHeaderAndValue(String header, String value) -> boolean
|
||||
hasRowWithHeaderAndAnyValue(String header, List<String> values) -> boolean
|
||||
getEntitiesOfTypeInSameRow(String type, TextEntity textEntity) -> List<TextEntity>
|
||||
getNumberOfRows() -> int
|
||||
getNumberOfCols() -> int
|
||||
|
||||
TableCell:
|
||||
getRow() -> int
|
||||
getCol() -> int
|
||||
isHeader() -> boolean
|
||||
|
||||
TextEntity:
|
||||
type() -> String
|
||||
value() -> String
|
||||
getEntityType() -> EntityType
|
||||
removed() -> boolean
|
||||
skipped() -> boolean
|
||||
active() -> boolean
|
||||
applied() -> boolean
|
||||
ignored() -> boolean
|
||||
getId() -> String
|
||||
legalBasis() -> String
|
||||
references() -> Set<TextEntity>
|
||||
length() -> int
|
||||
isType(String type) -> boolean
|
||||
getMatchedRule() -> MatchedRule
|
||||
getDeepestFullyContainingNode() -> SemanticNode
|
||||
getTextRange() -> TextRange
|
||||
contains(TextEntity textEntity) -> boolean
|
||||
containedBy(TextEntity textEntity) -> boolean
|
||||
intersects(TextEntity textEntity) -> boolean
|
||||
occursInNodeOfType(Class<? extends SemanticNode> clazz) -> boolean
|
||||
occursInNode(SemanticNode semanticNode) -> boolean
|
||||
isAnyType(List<String> types) -> boolean
|
||||
isDictionaryEntry() -> boolean
|
||||
isDossierDictionaryEntry() -> boolean
|
||||
getEngines() -> Set<Engine>
|
||||
getTextBefore() -> String
|
||||
getTextAfter() -> String
|
||||
getPages() -> Set<Page>
|
||||
getIntersectingNodes() -> List<SemanticNode>
|
||||
apply(String ruleIdentifier, String reason) -> void
|
||||
apply(String ruleIdentifier, String reason, String legalBasis) -> void
|
||||
redact(String ruleIdentifier, String reason, String legalBasis) -> void
|
||||
skip(String ruleIdentifier, String reason) -> void
|
||||
ignore(String ruleIdentifier, String reason) -> void
|
||||
remove(String ruleIdentifier, String reason) -> void
|
||||
applyWithLineBreaks(String ruleIdentifier, String reason, String legalBasis) -> void
|
||||
applyWithReferences(String ruleIdentifier, String reason, String legalBasis, Collection<TextEntity> references) -> void
|
||||
skipWithReferences(String ruleIdentifier, String reason, Collection<TextEntity> references) -> void
|
||||
|
||||
EntityType:
|
||||
ENTITY,
|
||||
HINT,
|
||||
RECOMMENDATION,
|
||||
FALSE_POSITIVE,
|
||||
FALSE_RECOMMENDATION
|
||||
|
||||
TextRange:
|
||||
length() -> int
|
||||
end() -> int
|
||||
start() -> int
|
||||
contains(int start, int end) -> boolean
|
||||
contains(int index) -> boolean
|
||||
contains(TextRange textRange) -> boolean
|
||||
merge(Collection<TextRange> textRanges) -> TextRange
|
||||
containedBy(int start, int end) -> boolean
|
||||
containedBy(TextRange textRange) -> boolean
|
||||
intersects(TextRange textRange) -> boolean
|
||||
""";
|
||||
|
||||
}
|
||||
@ -0,0 +1,7 @@
|
||||
package com.knecon.fforesight.llm.service.permissions;
|
||||
|
||||
public class LlmServicePermissions {
|
||||
|
||||
public static final String LLM = "fforesight-llm";
|
||||
|
||||
}
|
||||
@ -0,0 +1,27 @@
|
||||
package com.knecon.fforesight.llm.service.queue;
|
||||
|
||||
import org.springframework.amqp.core.Message;
|
||||
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
|
||||
import org.springframework.amqp.rabbit.annotation.RabbitListener;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.SneakyThrows;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class MessageHandler {
|
||||
|
||||
|
||||
|
||||
@SneakyThrows
|
||||
@RabbitHandler
|
||||
@RabbitListener(queues = "#{llmServiceSettings.getRequestQueueName()}")
|
||||
public void receiveIndexingRequest(Message message) {
|
||||
|
||||
// TODO: Do something.
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,11 @@
|
||||
package com.knecon.fforesight.llm.service.queue;
|
||||
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
@Configuration
|
||||
@RequiredArgsConstructor
|
||||
public class MessagingConfiguration {
|
||||
|
||||
}
|
||||
@ -0,0 +1,70 @@
|
||||
package com.knecon.fforesight.llm.service.services;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import org.springframework.messaging.simp.SimpMessagingTemplate;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.azure.ai.openai.OpenAIAsyncClient;
|
||||
import com.azure.ai.openai.OpenAIClientBuilder;
|
||||
import com.azure.ai.openai.models.ChatCompletions;
|
||||
import com.azure.ai.openai.models.ChatCompletionsOptions;
|
||||
import com.azure.ai.openai.models.ChatMessage;
|
||||
import com.azure.ai.openai.models.ChatRole;
|
||||
import com.azure.core.credential.AzureKeyCredential;
|
||||
import com.knecon.fforesight.keycloakcommons.security.KeycloakSecurity;
|
||||
import com.knecon.fforesight.llm.service.api.model.ChatEvent;
|
||||
import com.knecon.fforesight.llm.service.model.SystemMessages;
|
||||
import com.knecon.fforesight.llm.service.settings.LlmServiceSettings;
|
||||
import com.knecon.fforesight.tenantcommons.TenantContext;
|
||||
|
||||
import jakarta.annotation.PostConstruct;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.SneakyThrows;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import reactor.core.publisher.Flux;
|
||||
|
||||
@Slf4j
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class LlmService {
|
||||
|
||||
private final SimpMessagingTemplate websocketTemplate;
|
||||
private final LlmServiceSettings settings;
|
||||
private OpenAIAsyncClient client;
|
||||
|
||||
|
||||
@PostConstruct
|
||||
public void init() {
|
||||
|
||||
client = new OpenAIClientBuilder().credential(new AzureKeyCredential(settings.getAzureOpenAiKey())).endpoint(settings.getAzureOpenAiEndpoint()).buildAsyncClient();
|
||||
}
|
||||
|
||||
|
||||
@SneakyThrows
|
||||
public void execute(List<String> prompt) {
|
||||
|
||||
List<ChatMessage> chatMessages = new ArrayList<>();
|
||||
chatMessages.add(new ChatMessage(ChatRole.SYSTEM, SystemMessages.RULES_CO_PILOT));
|
||||
chatMessages.addAll(prompt.stream()
|
||||
.map(p -> new ChatMessage(ChatRole.USER, p))
|
||||
.toList());
|
||||
ChatCompletionsOptions options = new ChatCompletionsOptions(chatMessages);
|
||||
options.setStream(true);
|
||||
Flux<ChatCompletions> chatCompletions = client.getChatCompletionsStream(settings.getModel(), options);
|
||||
chatCompletions.subscribe(chatCompletion -> {
|
||||
sendWebsocketEvent(chatCompletion.getChoices()
|
||||
.get(0).getDelta().getContent());
|
||||
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
private void sendWebsocketEvent(String token) {
|
||||
|
||||
System.out.println("/topic/" + TenantContext.getTenantId() + "/chat-events/" + KeycloakSecurity.getUserId());
|
||||
websocketTemplate.convertAndSend("/topic/" + TenantContext.getTenantId() + "/chat-events/" + KeycloakSecurity.getUserId(), new ChatEvent(token));
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,25 @@
|
||||
package com.knecon.fforesight.llm.service.settings;
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
@Primary
|
||||
@Configuration
|
||||
@ConfigurationProperties("llm-service")
|
||||
public class LlmServiceSettings {
|
||||
|
||||
private String requestQueueName = "llm_request_queue";
|
||||
private String responseQueueName = "llm_response_queue";
|
||||
private String deadLetterQueueName = "llm_dead_letter_queue";
|
||||
|
||||
|
||||
private String azureOpenAiKey;
|
||||
private String azureOpenAiEndpoint;
|
||||
private String model = "gpt-4-cqs-dev";
|
||||
|
||||
|
||||
}
|
||||
@ -0,0 +1,97 @@
|
||||
package com.knecon.fforesight.llm.service.websocket;
|
||||
|
||||
import java.util.Collections;
|
||||
import java.util.Optional;
|
||||
|
||||
import org.apache.tomcat.websocket.server.WsSci;
|
||||
import org.springframework.boot.web.embedded.tomcat.TomcatContextCustomizer;
|
||||
import org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.messaging.Message;
|
||||
import org.springframework.messaging.MessageChannel;
|
||||
import org.springframework.messaging.simp.config.ChannelRegistration;
|
||||
import org.springframework.messaging.simp.config.MessageBrokerRegistry;
|
||||
import org.springframework.messaging.simp.stomp.StompCommand;
|
||||
import org.springframework.messaging.simp.stomp.StompHeaderAccessor;
|
||||
import org.springframework.messaging.support.ChannelInterceptor;
|
||||
import org.springframework.messaging.support.MessageHeaderAccessor;
|
||||
import org.springframework.security.authentication.AuthenticationManager;
|
||||
import org.springframework.security.oauth2.server.resource.authentication.BearerTokenAuthenticationToken;
|
||||
import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken;
|
||||
import org.springframework.web.socket.config.annotation.EnableWebSocketMessageBroker;
|
||||
import org.springframework.web.socket.config.annotation.StompEndpointRegistry;
|
||||
import org.springframework.web.socket.config.annotation.WebSocketMessageBrokerConfigurer;
|
||||
|
||||
import com.knecon.fforesight.keycloakcommons.security.TenantAuthenticationManagerResolver;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@Configuration
|
||||
@EnableWebSocketMessageBroker
|
||||
@RequiredArgsConstructor
|
||||
public class WebSocketConfiguration implements WebSocketMessageBrokerConfigurer {
|
||||
|
||||
private final TenantAuthenticationManagerResolver tenantAuthenticationManagerResolver;
|
||||
|
||||
|
||||
@Override
|
||||
public void configureMessageBroker(MessageBrokerRegistry config) {
|
||||
|
||||
config.setPreservePublishOrder(true);
|
||||
config.enableSimpleBroker("/topic");
|
||||
config.setApplicationDestinationPrefixes("/app");
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void registerStompEndpoints(StompEndpointRegistry registry) {
|
||||
|
||||
|
||||
registry.addEndpoint("/api/llm/llm-websocket").setAllowedOrigins("*");
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public void configureClientInboundChannel(ChannelRegistration registration) {
|
||||
|
||||
// https://docs.spring.io/spring-framework/reference/web/websocket/stomp/authentication-token-based.html
|
||||
registration.interceptors(new ChannelInterceptor() {
|
||||
@Override
|
||||
public Message<?> preSend(Message<?> message, MessageChannel channel) {
|
||||
|
||||
StompHeaderAccessor accessor = MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);
|
||||
if (StompCommand.CONNECT.equals(accessor.getCommand())) {
|
||||
Optional.ofNullable(accessor.getNativeHeader("Authorization"))
|
||||
.ifPresent(ah -> {
|
||||
String bearerToken = ah.get(0).replace("Bearer ", "");
|
||||
log.debug("Received bearer token {}", bearerToken);
|
||||
AuthenticationManager authenticationManager = tenantAuthenticationManagerResolver.resolve(bearerToken);
|
||||
JwtAuthenticationToken token = (JwtAuthenticationToken) authenticationManager.authenticate(new BearerTokenAuthenticationToken(bearerToken));
|
||||
accessor.setUser(token);
|
||||
});
|
||||
}
|
||||
return message;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@Bean
|
||||
public TomcatServletWebServerFactory tomcatContainerFactory() {
|
||||
|
||||
TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
|
||||
factory.setTomcatContextCustomizers(Collections.singletonList(tomcatContextCustomizer()));
|
||||
return factory;
|
||||
}
|
||||
|
||||
|
||||
@Bean
|
||||
public TomcatContextCustomizer tomcatContextCustomizer() {
|
||||
|
||||
return context -> context.addServletContainerInitializer(new WsSci(), null);
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,88 @@
|
||||
package com.knecon.fforesight.llm.service.websocket;
|
||||
|
||||
import java.util.Optional;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.messaging.Message;
|
||||
import org.springframework.messaging.simp.SimpMessageType;
|
||||
import org.springframework.messaging.simp.stomp.StompHeaderAccessor;
|
||||
import org.springframework.security.config.annotation.web.messaging.MessageSecurityMetadataSourceRegistry;
|
||||
import org.springframework.security.config.annotation.web.socket.AbstractSecurityWebSocketMessageBrokerConfigurer;
|
||||
import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken;
|
||||
|
||||
import com.knecon.fforesight.keycloakcommons.security.TokenUtils;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Slf4j
|
||||
@Configuration
|
||||
public class WebSocketSecurityConfig extends AbstractSecurityWebSocketMessageBrokerConfigurer {
|
||||
|
||||
@Value("${cors.enabled:false}")
|
||||
private boolean corsEnabled;
|
||||
|
||||
|
||||
@Override
|
||||
protected void configureInbound(MessageSecurityMetadataSourceRegistry messages) {
|
||||
|
||||
messages.simpTypeMatchers(SimpMessageType.HEARTBEAT, SimpMessageType.UNSUBSCRIBE, SimpMessageType.DISCONNECT)
|
||||
.permitAll()
|
||||
.simpTypeMatchers(SimpMessageType.CONNECT)
|
||||
.anonymous() // this is intended, see WebSocketConfiguration.configureClientInboundChannel
|
||||
.simpTypeMatchers(SimpMessageType.SUBSCRIBE)
|
||||
.access("@tenantWebSocketSecurityMatcher.checkCanSubscribeTo(authentication,message)")
|
||||
.anyMessage()
|
||||
.denyAll();
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
protected boolean sameOriginDisabled() {
|
||||
|
||||
return corsEnabled;
|
||||
}
|
||||
|
||||
|
||||
@Bean
|
||||
public TenantWebSocketSecurityMatcher tenantWebSocketSecurityMatcher() {
|
||||
|
||||
return new TenantWebSocketSecurityMatcher();
|
||||
}
|
||||
|
||||
|
||||
public class TenantWebSocketSecurityMatcher {
|
||||
|
||||
public boolean checkCanSubscribeTo(JwtAuthenticationToken authentication, Message<?> message) {
|
||||
|
||||
var targetedTenant = extractTenantId(message);
|
||||
var currentTenant = getCurrentTenant(authentication);
|
||||
return targetedTenant.isPresent() && currentTenant.isPresent() && currentTenant.get().equals(targetedTenant.get());
|
||||
}
|
||||
|
||||
|
||||
private Optional<String> getCurrentTenant(JwtAuthenticationToken authentication) {
|
||||
|
||||
if (authentication != null && authentication.getToken() != null && authentication.getToken().getTokenValue() != null) {
|
||||
return Optional.of(TokenUtils.toTenant(authentication.getToken().getTokenValue()));
|
||||
} else {
|
||||
return Optional.empty();
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
private Optional<String> extractTenantId(Message<?> message) {
|
||||
|
||||
StompHeaderAccessor sha = StompHeaderAccessor.wrap(message);
|
||||
String topic = sha.getDestination();
|
||||
if (topic == null) {
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
String tenant = topic.split("/")[2];
|
||||
return Optional.of(tenant);
|
||||
}
|
||||
|
||||
}
|
||||
65
src/main/resources/application.yaml
Normal file
65
src/main/resources/application.yaml
Normal file
@ -0,0 +1,65 @@
|
||||
spring:
|
||||
mvc:
|
||||
pathmatch:
|
||||
matching-strategy: ant-path-matcher
|
||||
async:
|
||||
request-timeout: 120s
|
||||
|
||||
rabbitmq:
|
||||
host: ${RABBITMQ_HOST:localhost}
|
||||
port: ${RABBITMQ_PORT:5672}
|
||||
username: ${RABBITMQ_USERNAME:user}
|
||||
password: ${RABBITMQ_PASSWORD:rabbitmq}
|
||||
listener:
|
||||
simple:
|
||||
acknowledge-mode: AUTO
|
||||
concurrency: 5
|
||||
retry:
|
||||
enabled: true
|
||||
max-attempts: 3
|
||||
max-interval: 15000
|
||||
prefetch: 1
|
||||
|
||||
llm-service:
|
||||
azureOpenAiKey: "Your Azure open Api Key"
|
||||
azureOpenAiEndpoint: "Your Azure open Api Endpoint"
|
||||
|
||||
fforesight:
|
||||
llm-service:
|
||||
base-path: '/api/llm'
|
||||
keycloak:
|
||||
ignored-endpoints: [ '/actuator/health', '/actuator/health/**', '/api/llm', '/api/llm/', '/internal/**', '/api/llm/docs/**', '/api/llm/docs', '/api/llm/chat-async', '/api/llm/llm-websocket' ]
|
||||
enabled: true
|
||||
springdoc:
|
||||
base-path: '/api/llm'
|
||||
auth-server-url: '/auth'
|
||||
enabled: true
|
||||
default-client-id: 'swagger-ui-client'
|
||||
default-tenant: 'fforesight'
|
||||
tenants:
|
||||
remote: true
|
||||
|
||||
springdoc:
|
||||
swagger-ui:
|
||||
path: ${fforesight.springdoc.base-path}/docs/swagger-ui
|
||||
operations-sorter: alpha
|
||||
tags-sorter: alpha
|
||||
oauth:
|
||||
client-id: swagger-ui-client
|
||||
doc-expansion: none
|
||||
config-url: ${fforesight.springdoc.base-path}/docs/swagger-config
|
||||
api-docs:
|
||||
path: ${fforesight.springdoc.base-path}/docs?tenantId=${fforesight.springdoc.default-tenant}
|
||||
enabled: ${fforesight.springdoc.enabled}
|
||||
pre-loading-enabled: true
|
||||
packages-to-scan: [ 'com.knecon.fforesight.llm.service.controller.external' ]
|
||||
tenant-user-management-service:
|
||||
url: "http://tenant-user-management-service:8080/internal"
|
||||
|
||||
text-analysis-service:
|
||||
url: "http://embedding-service:8080"
|
||||
|
||||
keyword-service:
|
||||
url: "http://keyword-extraction-service:8080"
|
||||
|
||||
cors.enabled: true
|
||||
1
src/main/resources/testcontainers.properties
Normal file
1
src/main/resources/testcontainers.properties
Normal file
@ -0,0 +1 @@
|
||||
hub.image.name.prefix=docker-dev.knecon.com/tests/
|
||||
@ -0,0 +1,83 @@
|
||||
package com.knecon.fforesight.llm.service;
|
||||
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.extension.ExtendWith;
|
||||
import org.springframework.amqp.rabbit.core.RabbitTemplate;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||
import org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration;
|
||||
import org.springframework.boot.test.autoconfigure.actuate.observability.AutoConfigureObservability;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.mock.mockito.MockBean;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.ComponentScan;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.context.annotation.FilterType;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
import org.springframework.test.annotation.DirtiesContext;
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.iqser.red.storage.commons.StorageAutoConfiguration;
|
||||
import com.iqser.red.storage.commons.service.StorageService;
|
||||
import com.iqser.red.storage.commons.utils.FileSystemBackedStorageService;
|
||||
import com.knecon.fforesight.tenantcommons.TenantContext;
|
||||
import com.knecon.fforesight.tenantcommons.TenantsClient;
|
||||
import com.knecon.fforesight.tenantcommons.model.TenantResponse;
|
||||
|
||||
@ComponentScan
|
||||
@ExtendWith(SpringExtension.class)
|
||||
@SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
|
||||
@Import(AbstractLlmServiceIntegrationTest.TestConfiguration.class)
|
||||
@DirtiesContext
|
||||
@AutoConfigureObservability
|
||||
public abstract class AbstractLlmServiceIntegrationTest {
|
||||
|
||||
public static final String TEST_TENANT = "test";
|
||||
@Autowired
|
||||
protected StorageService storageService;
|
||||
@MockBean
|
||||
TenantsClient tenantsClient;
|
||||
@MockBean
|
||||
RabbitTemplate rabbitTemplate;
|
||||
|
||||
|
||||
@BeforeEach
|
||||
public void setupOptimize() {
|
||||
|
||||
var tenant = TenantResponse.builder()
|
||||
.tenantId(TEST_TENANT)
|
||||
.build();
|
||||
|
||||
TenantContext.setTenantId(TEST_TENANT);
|
||||
|
||||
when(tenantsClient.getTenant(TEST_TENANT)).thenReturn(tenant);
|
||||
when(tenantsClient.getTenants()).thenReturn(List.of(tenant));
|
||||
}
|
||||
|
||||
|
||||
@SuppressWarnings("PMD.TestClassWithoutTestCases")
|
||||
@Configuration
|
||||
@EnableAutoConfiguration(exclude = {RabbitAutoConfiguration.class})
|
||||
@ComponentScan(excludeFilters = {@ComponentScan.Filter(type = FilterType.ASSIGNABLE_TYPE, value = StorageAutoConfiguration.class)})
|
||||
public static class TestConfiguration {
|
||||
|
||||
@Autowired
|
||||
private ObjectMapper objectMapper;
|
||||
|
||||
|
||||
@Bean
|
||||
@Primary
|
||||
public StorageService inmemoryStorage() {
|
||||
|
||||
return new FileSystemBackedStorageService(objectMapper);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
package com.knecon.fforesight.llm.service;
|
||||
|
||||
import static org.junit.jupiter.api.Assumptions.assumeTrue;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
public class IdentityTest {
|
||||
|
||||
@Test
|
||||
public void test() {
|
||||
|
||||
assumeTrue(true);
|
||||
}
|
||||
|
||||
}
|
||||
@ -0,0 +1,22 @@
|
||||
package com.knecon.fforesight.llm.service;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import org.junit.jupiter.api.Disabled;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
import com.knecon.fforesight.llm.service.services.LlmService;
|
||||
|
||||
@Disabled
|
||||
public class LlmServiceIntegrationTest extends AbstractLlmServiceIntegrationTest{
|
||||
|
||||
@Autowired
|
||||
private LlmService llmService;
|
||||
|
||||
@Test
|
||||
public void testLlm(){
|
||||
llmService.execute(List.of("Wie backe ich eine tiefkühl Pizza?"));
|
||||
}
|
||||
|
||||
}
|
||||
17
src/test/resources/application.yaml
Normal file
17
src/test/resources/application.yaml
Normal file
@ -0,0 +1,17 @@
|
||||
server:
|
||||
port: 28080
|
||||
fforesight:
|
||||
keycloak:
|
||||
enabled: false
|
||||
springdoc:
|
||||
enabled: false
|
||||
|
||||
tenant-user-management-service.url: "http://mock.url"
|
||||
text-analysis-service.url: "http://mock.url"
|
||||
epam-poc-service.url: "http://mock.url"
|
||||
keyword-service.url: "http://mock.url"
|
||||
|
||||
|
||||
llm-service:
|
||||
azureOpenAiKey: "Your Azure open Api Key"
|
||||
azureOpenAiEndpoint: "Your Azure open Api Endpoint"
|
||||
Loading…
x
Reference in New Issue
Block a user