Compare commits

...

72 Commits

Author SHA1 Message Date
Kilian Schüttler
418ac81465 Merge branch 'RM-226' into 'master'
RM-226: chunk empty string correctly

Closes RM-226

See merge request redactmanager/redaction-report-service!115
2025-01-21 17:00:46 +01:00
Kilian Schüttler
7f145eb2e7 RM-226: chunk empty string correctly 2025-01-21 17:00:45 +01:00
Kilian Schüttler
a6154b7b80 Merge branch 'RM-226' into 'master'
RM-226: write empty components

Closes RM-226

See merge request redactmanager/redaction-report-service!114
2025-01-21 14:53:06 +01:00
Kilian Schüttler
acf4683cfe RM-226: write empty components 2025-01-21 14:53:05 +01:00
Kilian Schüttler
0de5f53cb9 Merge branch 'RM-226' into 'master'
RM-226: Column headers missing

Closes RM-226

See merge request redactmanager/redaction-report-service!112
2025-01-20 12:25:14 +01:00
Kilian Schuettler
7f9b386d35 RM-226: Column headers missing 2025-01-20 12:14:02 +01:00
Kilian Schüttler
c0fd79d4d8 Merge branch 'RED-10683' into 'master'
RED-10683: fix NPE for multi file reports

Closes RED-10683

See merge request redactmanager/redaction-report-service!111
2024-12-16 16:30:47 +01:00
Kilian Schuettler
e39858b51a RED-10683: fix NPE for multi file reports 2024-12-16 16:23:22 +01:00
Corina Olariu
7c15154ab8 Merge branch 'feature/RED-10553' into 'master'
RED-10553 - New report placeholder for section headline

Closes RED-10553

See merge request redactmanager/redaction-report-service!109
2024-12-16 11:02:10 +01:00
corinaolariu
41b8e8b821 RED-10553 - New report placeholder for section headline
- add new placeholder: headline {{redaction.entity.closestHeadline}} to excel reports
2024-12-09 14:55:10 +02:00
Maverick Studer
2b565fd523 Merge branch 'feature/RED-10115' into 'master'
RED-10115: Refactoring of justifications

Closes RED-10115

See merge request redactmanager/redaction-report-service!106
2024-11-20 10:44:53 +01:00
Maverick Studer
ff0592c7f6 RED-10115: Refactoring of justifications 2024-11-20 10:44:52 +01:00
Maverick Studer
8b09a07264 Merge branch 'RED-10435' into 'master'
RED-10435: RM-216: Report generation fails when component is to long

Closes RED-10435

See merge request redactmanager/redaction-report-service!104
2024-11-11 12:03:25 +01:00
maverickstuder
bb0bbe7fb6 RED-10435: RM-216: Report generation fails when component is to long 2024-11-11 11:59:43 +01:00
Dominique Eifländer
e51fc5eb74 Merge branch 'RED-10289' into 'master'
RED-10289: Fixed image placeholders are only replaced in tables

Closes RED-10289

See merge request redactmanager/redaction-report-service!103
2024-11-04 09:48:02 +01:00
Dominique Eifländer
20fdcf4c9a RED-10289: Fixed image placeholders are only replaced in tables 2024-11-04 09:44:46 +01:00
Dominique Eifländer
abcdb63152 Merge branch 'RED-10289' into 'master'
RED-10289: Fixed png image placeholders

Closes RED-10289

See merge request redactmanager/redaction-report-service!102
2024-10-29 12:09:00 +01:00
Dominique Eifländer
e5b198bb3f RED-10289: Fixed png image placeholders 2024-10-29 10:38:25 +01:00
Dominique Eifländer
f4506557a1 Merge branch 'RED-10094' into 'master'
RED-10094: Revert back to not use justification technical name for lookup...

Closes RED-10094

See merge request redactmanager/redaction-report-service!100
2024-09-24 15:52:15 +02:00
Dominique Eifländer
69c989c3a4 RED-10094: Revert back to not use justification technical name for lookup justification reason, as technical name feature is too incomplte for release 2024-09-24 15:16:52 +02:00
Maverick Studer
31b793f0a2 Merge branch 'feature/RED-9348' into 'master'
RED-9348: move component log to mongodb

Closes RED-9348

See merge request redactmanager/redaction-report-service!97
2024-09-23 16:05:06 +02:00
Maverick Studer
ff10bf07c9 RED-9348: move component log to mongodb 2024-09-23 16:05:05 +02:00
Andrei Isvoran
21775d5a7d Merge branch 'RED-9925-fixes' into 'master'
RED-9925 - Remove extra breakline from report

Closes RED-9925

See merge request redactmanager/redaction-report-service!99
2024-09-20 14:00:14 +02:00
Andrei Isvoran
bf6627ed5d RED-9925 - Remove extra breakline from report 2024-09-20 14:15:40 +03:00
Andrei Isvoran
d03815a7a1 Merge branch 'RED-9925' into 'master'
RED-9925 - Remove extra breakline from report

Closes RED-9925

See merge request redactmanager/redaction-report-service!98
2024-09-19 16:08:51 +02:00
Andrei Isvoran
41ac300907 RED-9925 - Remove extra breakline from report 2024-09-19 15:27:44 +03:00
Maverick Studer
2059e195de Merge branch 'update-tc' into 'master'
Update tenant-commons for dlq fix

See merge request redactmanager/redaction-report-service!96
2024-09-03 13:49:58 +02:00
maverickstuder
5e892e04c3 Update tenant-commons for dlq fix 2024-09-03 13:17:29 +02:00
Maverick Studer
400c4dfed6 Merge branch 'tenants-retry' into 'master'
Tenants retry logic and queue renames

See merge request redactmanager/redaction-report-service!95
2024-08-29 16:12:40 +02:00
Maverick Studer
de3817fd50 Tenants retry logic and queue renames 2024-08-29 16:12:39 +02:00
Maverick Studer
bb87355d7a Merge branch 'REVERT-RED-7327' into 'master'
Revert "RED-7327 - Add group redaction"

Closes RED-7327

See merge request redactmanager/redaction-report-service!94
2024-08-29 10:18:53 +02:00
Maverick Studer
8bdfe874ab Revert "RED-7327 - Add group redaction" 2024-08-29 10:18:53 +02:00
Maverick Studer
10848f43c7 Merge branch 'RED-9331' into 'master'
RED-9331: Explore possibilities for fair upload / analysis processing per tenant

Closes RED-9331

See merge request redactmanager/redaction-report-service!89
2024-08-27 13:23:40 +02:00
Maverick Studer
c467944383 RED-9331: Explore possibilities for fair upload / analysis processing per tenant 2024-08-27 13:23:40 +02:00
Andrei Isvoran
af392be6c7 Merge branch 'RED-7327' into 'master'
RED-7327 - Add group redaction

Closes RED-7327

See merge request redactmanager/redaction-report-service!93
2024-08-20 10:32:05 +02:00
Andrei Isvoran
8e42c783d0 RED-7327 - Add group redaction 2024-08-20 10:32:04 +02:00
Maverick Studer
1ecd05f91c Merge branch 'RED-9782' into 'master'
RED-9782:Automated Analysis should be disabled when uploading a document that...

Closes RED-9782

See merge request redactmanager/redaction-report-service!91
2024-08-12 18:41:08 +02:00
maverickstuder
876bef2e00 RED-9782:Automated Analysis should be disabled when uploading a document that contains redactions from RedactManager
* fix for findings
2024-08-12 14:50:47 +02:00
Yannik Hampe
673e754430 Merge branch 'RED-5624' into 'master'
RED-5624: Refactoring of justifications

Closes RED-5624

See merge request redactmanager/redaction-report-service!90
2024-08-12 11:03:02 +02:00
yhampe
ddc16419e9 RED-5624: Refactoring of justifications
readded decription to reports
changed filter to filter out null technicalnames
2024-08-12 09:42:42 +02:00
yhampe
030aafcc5f RED-5624: Refactoring of justifications
added technicalname instead of reason to reports
2024-08-09 14:02:56 +02:00
yhampe
6b2118e803 RED-5624: Refactoring of justifications
added technicalname instead of reason to reports
2024-08-09 10:13:43 +02:00
yhampe
307e51b97b RED-5624: Refactoring of justifications
added technicalname instead of reason to reports
2024-08-08 09:54:08 +02:00
yhampe
86feb15ee9 RED-5624: Refactoring of justifications
added technicalname instead of reason to reports
2024-08-07 09:39:50 +02:00
Andrei Isvoran
049c01dcec Merge branch 'RED-9579' into 'master'
RED-9579 - Use TaskExecutor so the TenantAwareTaskDecorator will correctly set...

Closes RED-9579

See merge request redactmanager/redaction-report-service!85
2024-07-11 09:01:58 +02:00
Andrei Isvoran
0ba434a48d RED-9579 - Use TaskExecutor so the TenantAwareTaskDecorator will correctly set... 2024-07-11 09:01:57 +02:00
Ali Oezyetimoglu
604ea54353 Merge branch 'RED-9517-dom' into 'master'
RED-9517: Use redis for chaching report templates

Closes RED-9517

See merge request redactmanager/redaction-report-service!82
2024-07-10 12:48:12 +02:00
Dominique Eifländer
ab6defc225 RED-9517: Use redis for chaching report templates 2024-07-05 11:35:18 +02:00
Andrei Isvoran
88279326aa Merge branch 'RED-9496-lifecycle' into 'master'
RED-9496 - Implement graceful shutdown

Closes RED-9496

See merge request redactmanager/redaction-report-service!82
2024-07-04 08:42:46 +02:00
Andrei Isvoran
08eab315c4 RED-9496 - Implement graceful shutdown 2024-07-03 17:29:41 +03:00
Andrei Isvoran
6b322e4ea6 Merge branch 'RED-9496' into 'master'
RED-9496 - Implement graceful shutdown

Closes RED-9496

See merge request redactmanager/redaction-report-service!81
2024-07-03 09:32:16 +02:00
Andrei Isvoran
1d54130398 RED-9496 - Implement graceful shutdown 2024-07-03 09:32:16 +02:00
Andrei Isvoran
3dea3e77f5 Merge branch 'RED-9482-pending' into 'master'
RED-9482 - Filter out pending entries from the report

Closes RED-9482

See merge request redactmanager/redaction-report-service!80
2024-06-28 14:34:37 +02:00
Andrei Isvoran
917fe5433b RED-9482 - Filter out pending entries from the report 2024-06-28 14:47:39 +03:00
Andrei Isvoran
f0a279d25b Merge branch 'RED-9140' into 'master'
RED-9140 - Bump persistence service version

Closes RED-9140

See merge request redactmanager/redaction-report-service!77
2024-06-27 16:55:52 +02:00
Andrei Isvoran
38cc2bb536 RED-9140 - Bump persistence service version 2024-06-27 17:46:15 +03:00
Corina Olariu
1290f0dab6 Merge branch 'RED-9347' into 'master'
RED-9347 - Make /rss controller unavailable

Closes RED-9347

See merge request redactmanager/redaction-report-service!76
2024-06-27 07:18:11 +02:00
corinaolariu
110ed39fc4 RED-9347 - Make /rss controller unavailable
- delete unused endpoints (/rss) anymore
2024-06-26 11:57:32 +03:00
Corina Olariu
679ae58ba0 Merge branch 'RED-9409' into 'master'
RED-9409 - Excel QC includes hints and big merged empty space

Closes RED-9409

See merge request redactmanager/redaction-report-service!75
2024-06-24 12:38:46 +02:00
Corina Olariu
9a0156bcd4 RED-9409 - Excel QC includes hints and big merged empty space
- update junit test to solve sonar issues
2024-06-24 08:46:50 +03:00
Corina Olariu
2ba5c7427d RED-9409 - Excel QC includes hints and big merged empty space
- update junit test
2024-06-21 20:46:28 +03:00
Corina Olariu
3cded28f40 RED-9409 - Excel QC includes hints and big merged empty space
- filter the report entries so that entry type HINT is not included, entry type IMAGE_HINT with state skipped is not included and also add entry type AREA with states skipped or ignored to not be included. All removed entry types are not present
- update the Excel QC template so that the rows after the placeholder row are not formatted
- add junit test
2024-06-21 20:34:05 +03:00
Ali Oezyetimoglu
553da64f6a Merge branch 'RED-9286' into 'master'
RED-9286: added test for checking for skipped values

Closes RED-9286

See merge request redactmanager/redaction-report-service!74
2024-06-13 11:13:59 +02:00
Ali Oezyetimoglu
9edf89d563 RED-9286: added test for checking for skipped values 2024-06-12 17:01:41 +02:00
Ali Oezyetimoglu
f62a3bc543 Merge branch 'RED-9286' into 'master'
RED-9286: while converting entity log entries to report entries, all entries...

Closes RED-9286

See merge request redactmanager/redaction-report-service!73
2024-06-10 14:46:24 +02:00
Ali Oezyetimoglu
a2baf31172 RED-9286: while converting entity log entries to reort entries, all entries which were not in state APPLIED were not converted 2024-06-10 13:44:56 +02:00
Maverick Studer
3f93cad493 Merge branch 'RED-5870' into 'master'
RED-5870: Enable force redaction of hints

Closes RED-5870

See merge request redactmanager/redaction-report-service!72
2024-05-23 10:45:44 +02:00
maverickstuder
8b06e491d2 RED-5870: Enable force redaction of hints
* removed exclusion of hints on conversion to ReportRedactionEntry
2024-05-23 09:15:16 +02:00
Andrei Isvoran
71a8f17a93 Merge branch 'RED-9148' into 'master'
RED-9148 - Paragraph information placeholder for reports

Closes RED-9148

See merge request redactmanager/redaction-report-service!71
2024-05-22 11:10:33 +02:00
Andrei Isvoran
0bc6725424 RED-9148 - Paragraph information placeholder for reports 2024-05-22 11:35:53 +03:00
Andrei Isvoran
855da70b93 Merge branch 'RED-9157-fonts' into 'master'
RED-9157 - rollback to older linux distribution for paketo bp

Closes RED-9157

See merge request redactmanager/redaction-report-service!70
2024-05-16 13:51:33 +02:00
Andrei Isvoran
945555f7ad RED-9157 - rollback to older linux distribution for paketo bp 2024-05-16 14:40:08 +03:00
82 changed files with 51180 additions and 41139 deletions

View File

@ -24,4 +24,5 @@ deploy:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_COMMIT_BRANCH =~ /^release/
- if: $CI_COMMIT_BRANCH =~ /^feature/
- if: $CI_COMMIT_TAG

View File

@ -11,5 +11,5 @@ commit_hash=$(git rev-parse --short=5 HEAD)
# Combine branch and commit hash
buildName="${USER}-${branch}-${commit_hash}"
gradle bootBuildImage --cleanCache --publishImage -PbuildbootDockerHostNetwork=true -Pversion=$buildName
gradle bootBuildImage --cleanCache --publishImage -Pversion=$buildName
echo "nexus.knecon.com:5001/red/${dir}-server-v1:$buildName"

48
publish-custom-image.sh Executable file
View File

@ -0,0 +1,48 @@
#!/bin/bash
set -e
dir=${PWD##*/}
gradle assemble
# Get the current Git branch
branch=$(git rev-parse --abbrev-ref HEAD)
# Replace any slashes (e.g., in 'feature/' or 'release/') with a hyphen
cleaned_branch=$(echo "$branch" | sed 's/\//_/g')
# Get the short commit hash (first 5 characters)
commit_hash=$(git rev-parse --short=5 HEAD)
# Combine branch and commit hash
buildName="${USER}-${cleaned_branch}-${commit_hash}"
gradle bootBuildImage --publishImage -PbuildbootDockerHostNetwork=true -Pversion=${buildName}
newImageName="nexus.knecon.com:5001/red/${dir}-server-v1:${buildName}"
echo "full image name:"
echo ${newImageName}
echo ""
if [ -z "$1" ]; then
exit 0
fi
namespace=${1}
deployment_name="redaction-report-service-v1"
echo "deploying to ${namespace}"
oldImageName=$(rancher kubectl -n ${namespace} get deployment ${deployment_name} -o=jsonpath='{.spec.template.spec.containers[*].image}')
if [ "${newImageName}" = "${oldImageName}" ]; then
echo "Image tag did not change, redeploying..."
rancher kubectl rollout restart deployment ${deployment_name} -n ${namespace}
else
echo "upgrading the image tag..."
rancher kubectl set image deployment/${deployment_name} ${deployment_name}=${newImageName} -n ${namespace}
fi
rancher kubectl rollout status deployment ${deployment_name} -n ${namespace}
echo "Built ${deployment_name}:${buildName} and deployed to ${namespace}"

View File

@ -5,7 +5,7 @@ plugins {
description = "redaction-report-service-api-v1"
val persistenceServiceVersion = "2.380.0"
val persistenceServiceVersion = "2.562.0-RED9348.2"
dependencies {
implementation("io.github.openfeign:feign-core:12.2")

View File

@ -1,28 +0,0 @@
package com.iqser.red.service.redaction.report.v1.api.resource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseStatus;
import com.iqser.red.service.redaction.report.v1.api.model.rss.DetailedRSSResponse;
import com.iqser.red.service.redaction.report.v1.api.model.rss.RSSResponse;
@ResponseStatus(value = HttpStatus.OK)
public interface RSSResource {
String RSS_PATH = "/rss";
String DOSSIER_ID = "dossierId";
String DOSSIER_ID_PATH_VARIABLE = "/{" + DOSSIER_ID + "}";
@GetMapping(value = RSS_PATH + DOSSIER_ID_PATH_VARIABLE, produces = {MediaType.APPLICATION_JSON_VALUE, MediaType.APPLICATION_XML_VALUE})
RSSResponse getRSS(@PathVariable(DOSSIER_ID) String dossierId, @RequestParam(value = "fileId", required = false) String fileId);
@GetMapping(value = RSS_PATH + "/detailed" + DOSSIER_ID_PATH_VARIABLE, produces = {MediaType.APPLICATION_JSON_VALUE, MediaType.APPLICATION_XML_VALUE})
DetailedRSSResponse getDetailedRSS(@PathVariable(DOSSIER_ID) String dossierId, @RequestParam(value = "fileId", required = false) String fileId);
}

View File

@ -14,11 +14,7 @@ public interface ReportTemplateResource {
String REPORT_TEMPLATE_PATH = "/report-templates";
String REPORT_TEMPLATE_UPLOAD_PATH = "/upload-template";
String TEMPLATE_ID = "templateId";
String TEMPLATE_ID_PATH_VARIABLE = "/template-id/{" + TEMPLATE_ID + "}";
String EVICT_REPORT_TEMPLATE_CACHE_PATH = "/evict-report-template-cache";
String DOSSIER_TEMPLATE_ID = "dossierTemplateId";
@ -29,7 +25,7 @@ public interface ReportTemplateResource {
List<ReportTemplate> getReportTemplatesByPlaceholder(@PathVariable(DOSSIER_TEMPLATE_ID) String dossierTemplateId, @RequestBody JSONPrimitive<String> placeholder);
@PostMapping(value = REPORT_TEMPLATE_UPLOAD_PATH + TEMPLATE_ID_PATH_VARIABLE)
void uploadTemplate(@PathVariable(TEMPLATE_ID) String templateId);
@PostMapping(value = EVICT_REPORT_TEMPLATE_CACHE_PATH)
void evictReportTemplateCache();
}

View File

@ -11,12 +11,13 @@ plugins {
description = "redaction-service-server-v1"
val tenantCommonVersion = "0.21.0"
val tenantCommonVersion = "0.30.0"
val springCommonsVersion = "2.1.0"
val storageCommonsVersion = "2.45.0"
val lifecycleCommonsVersion = "0.4.0"
val poiVersion = "5.2.3"
val metricCommonsVersion = "2.1.0"
val persistenceServiceVersion = "2.380.0"
val persistenceServiceVersion = "2.562.0-RED9348.2"
val springBootStarterVersion = "3.2.3"
configurations {
@ -32,11 +33,13 @@ description = "redaction-report-service-server-v1"
dependencies {
implementation(project(":redaction-report-service-api-v1"))
implementation("com.iqser.red.service:persistence-service-internal-api-v1:${persistenceServiceVersion}")
implementation("com.iqser.red.service:persistence-service-shared-mongo-v1:${persistenceServiceVersion}")
implementation("com.knecon.fforesight:tenant-commons:${tenantCommonVersion}")
implementation("com.iqser.red.commons:storage-commons:${storageCommonsVersion}")
implementation("com.iqser.red.commons:spring-commons:${springCommonsVersion}")
implementation("com.iqser.red.commons:metric-commons:${metricCommonsVersion}")
implementation("com.knecon.fforesight:lifecycle-commons:${lifecycleCommonsVersion}")
implementation("com.knecon.fforesight:tracing-commons:0.5.0")
implementation("org.apache.poi:poi:${poiVersion}")
@ -44,9 +47,13 @@ dependencies {
implementation("org.apache.poi:poi-scratchpad:${poiVersion}")
implementation("org.springframework.boot:spring-boot-starter-amqp:${springBootStarterVersion}")
implementation("io.github.openfeign:feign-core:12.2")
implementation("org.springframework.cloud:spring-cloud-starter-openfeign:4.1.1")
implementation("org.apache.commons:commons-lang3:3.12.0")
implementation("org.springframework.boot:spring-boot-starter-cache:${springBootStarterVersion}")
implementation("org.springframework.boot:spring-boot-starter-data-redis:${springBootStarterVersion}")
implementation("com.github.ben-manes.caffeine:caffeine:3.1.8")
implementation("net.logstash.logback:logstash-logback-encoder:7.4")
@ -69,9 +76,11 @@ tasks.named<BootBuildImage>("bootBuildImage") {
val aptFile = layout.projectDirectory.file("src/main/resources/Aptfile").toString()
bindings.add("${aptFile}:/workspace/Aptfile:ro")
builder.set("paketobuildpacks/builder:base-unsafe")
buildpacks.set(
listOf(
"ghcr.io/knsita/buildpacks/fagiani_apt@sha256:9771d4d27d8050aee62769490b8882fffc794745c129fb98e1f33196e2c93504",
"ghcr.io/fagiani/buildpacks/fagiani_apt@sha256:6471c8c70f32b749e29f65ae562ac0339fecad26aa9217628c00a6c31f197dae",
"urn:cnb:builder:paketo-buildpacks/java"
)
)

View File

@ -4,28 +4,39 @@ import org.springframework.boot.SpringApplication;
import org.springframework.boot.actuate.autoconfigure.security.servlet.ManagementWebSecurityAutoConfiguration;
import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.boot.autoconfigure.liquibase.LiquibaseAutoConfiguration;
import org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration;
import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.cloud.openfeign.EnableFeignClients;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.context.annotation.Import;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
import org.springframework.scheduling.annotation.EnableAsync;
import com.iqser.red.service.persistence.service.v1.api.shared.mongo.SharedMongoAutoConfiguration;
import com.iqser.red.service.redaction.report.v1.server.client.DossierClient;
import com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration;
import com.iqser.red.service.redaction.report.v1.server.settings.ReportTemplateSettings;
import com.iqser.red.storage.commons.StorageAutoConfiguration;
import com.knecon.fforesight.lifecyclecommons.LifecycleAutoconfiguration;
import com.knecon.fforesight.mongo.database.commons.MongoDatabaseCommonsAutoConfiguration;
import com.knecon.fforesight.tenantcommons.MultiTenancyAutoConfiguration;
import io.micrometer.core.aop.TimedAspect;
import io.micrometer.core.instrument.MeterRegistry;
@EnableAsync
@ImportAutoConfiguration({MultiTenancyAutoConfiguration.class})
@SpringBootApplication(exclude = {SecurityAutoConfiguration.class, ManagementWebSecurityAutoConfiguration.class})
@Import({MessagingConfiguration.class, StorageAutoConfiguration.class})
@ImportAutoConfiguration({MultiTenancyAutoConfiguration.class, SharedMongoAutoConfiguration.class})
@SpringBootApplication(exclude = {SecurityAutoConfiguration.class, ManagementWebSecurityAutoConfiguration.class, ManagementWebSecurityAutoConfiguration.class, DataSourceAutoConfiguration.class, LiquibaseAutoConfiguration.class, MongoAutoConfiguration.class, MongoDataAutoConfiguration.class})
@Import({MessagingConfiguration.class, StorageAutoConfiguration.class, MongoDatabaseCommonsAutoConfiguration.class, LifecycleAutoconfiguration.class})
@EnableFeignClients(basePackageClasses = {DossierClient.class})
@EnableMongoRepositories(basePackages = "com.iqser.red.service.persistence")
@EnableConfigurationProperties(ReportTemplateSettings.class)
@EnableAspectJAutoProxy
public class Application {
/**

View File

@ -0,0 +1,31 @@
package com.iqser.red.service.redaction.report.v1.server.cache;
import java.time.Duration;
import org.springframework.boot.autoconfigure.cache.RedisCacheManagerBuilderCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
@Configuration
public class RedisCachingConfiguration {
@Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder.withCacheConfiguration("reportTemplateCache",
RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofMinutes(30)).disableCachingNullValues());
}
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory);
return template;
}
}

View File

@ -0,0 +1,32 @@
package com.iqser.red.service.redaction.report.v1.server.cache;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@Slf4j
@Service
@RequiredArgsConstructor
public class ReportTemplateCache {
private final ReportStorageService reportStorageService;
@Cacheable(value = "reportTemplateCache")
public byte[] getTemplate(String storageId) {
log.info("Putting report template with storageId {} into cache", storageId);
return reportStorageService.getReportTemplate(storageId);
}
@CacheEvict(value = "reportTemplateCache", allEntries = true)
public void evictReportTemplateCache() {
log.info("Evicted cache for report templates");
}
}

View File

@ -1,5 +1,6 @@
package com.iqser.red.service.redaction.report.v1.server.configuration;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.QueueBuilder;
import org.springframework.context.annotation.Bean;
@ -11,43 +12,25 @@ import lombok.RequiredArgsConstructor;
@RequiredArgsConstructor
public class MessagingConfiguration {
public static final String REPORT_QUEUE = "reportQueue";
public static final String REPORT_DLQ = "reportDLQ";
public static final String REPORT_REQUEST_QUEUE_PREFIX = "report_request";
public static final String REPORT_REQUEST_EXCHANGE = "report_request_exchange";
public static final String REPORT_REQUEST_DLQ = "report_request_error";
public static final String REPORT_RESULT_QUEUE = "reportResultQueue";
public static final String REPORT_RESULT_DLQ = "reportResultDLQ";
public static final String REPORT_RESPONSE_EXCHANGE = "report_response_exchange";
public static final String REPORT_RESPONSE_DLQ = "report_response_error";
@Bean
public Queue reportQueue() {
public DirectExchange reportRequestExchange() {
return QueueBuilder.durable(REPORT_QUEUE)
.withArgument("x-dead-letter-exchange", "")
.withArgument("x-dead-letter-routing-key", REPORT_DLQ)
.withArgument("x-max-priority", 2)
.maxPriority(2)
.build();
return new DirectExchange(REPORT_REQUEST_EXCHANGE);
}
@Bean
public Queue reportDeadLetterQueue() {
public Queue reportRequestDLQ() {
return QueueBuilder.durable(REPORT_DLQ).build();
}
@Bean
public Queue reportResultQueue() {
return QueueBuilder.durable(REPORT_RESULT_QUEUE).withArgument("x-dead-letter-exchange", "").withArgument("x-dead-letter-routing-key", REPORT_RESULT_DLQ).build();
}
@Bean
public Queue reportResultDeadLetterQueue() {
return QueueBuilder.durable(REPORT_RESULT_DLQ).build();
return QueueBuilder.durable(REPORT_REQUEST_DLQ).build();
}
}

View File

@ -0,0 +1,10 @@
package com.iqser.red.service.redaction.report.v1.server.configuration;
import org.springframework.context.annotation.Configuration;
import com.knecon.fforesight.tenantcommons.queue.TenantMessagingConfiguration;
@Configuration
public class TenantMessagingConfigurationImpl extends TenantMessagingConfiguration {
}

View File

@ -1,56 +0,0 @@
package com.iqser.red.service.redaction.report.v1.server.controller;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import com.iqser.red.service.redaction.report.v1.api.model.rss.DetailedRSSFileResponse;
import com.iqser.red.service.redaction.report.v1.api.model.rss.DetailedRSSResponse;
import com.iqser.red.service.redaction.report.v1.api.model.rss.RSSFileResponse;
import com.iqser.red.service.redaction.report.v1.api.model.rss.RSSResponse;
import com.iqser.red.service.redaction.report.v1.api.model.rss.SCMComponent;
import com.iqser.red.service.redaction.report.v1.api.resource.RSSResource;
import com.iqser.red.service.redaction.report.v1.server.service.RSSPoc2Service;
import lombok.RequiredArgsConstructor;
@RestController
@RequiredArgsConstructor
public class RSSController implements RSSResource {
private final RSSPoc2Service rSSService;
public RSSResponse getRSS(@PathVariable(DOSSIER_ID) String dossierId, @RequestParam(value = "fileId", required = false) String fileId) {
var detailed = rSSService.getRSS(dossierId, fileId);
List<RSSFileResponse> fileResponses = new ArrayList<>();
for (DetailedRSSFileResponse detailedRSSFileResponse : detailed.getFiles()) {
String fileName = detailedRSSFileResponse.getFilename();
Map<String, String> result = new LinkedHashMap<>();
for (Map.Entry<String, SCMComponent> detailedRss : detailedRSSFileResponse.getResult().entrySet()) {
if (detailedRss.getValue().getValue() != null) {
result.put(detailedRss.getKey(), detailedRss.getValue().getValue());
} else {
result.put(detailedRss.getKey(), detailedRss.getValue().getOriginalValue());
}
}
fileResponses.add(new RSSFileResponse(fileName, result));
}
return new RSSResponse(fileResponses);
}
public DetailedRSSResponse getDetailedRSS(@PathVariable(DOSSIER_ID) String dossierId, @RequestParam(value = "fileId", required = false) String fileId) {
return rSSService.getRSS(dossierId, fileId);
}
}

View File

@ -10,7 +10,7 @@ import com.iqser.red.service.persistence.service.v1.api.shared.model.common.JSON
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.ReportTemplate;
import com.iqser.red.service.redaction.report.v1.api.resource.ReportTemplateResource;
import com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService;
import com.iqser.red.service.redaction.report.v1.server.utils.TemplateCache;
import com.iqser.red.service.redaction.report.v1.server.cache.ReportTemplateCache;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@ -21,6 +21,7 @@ import lombok.extern.slf4j.Slf4j;
public class ReportTemplateController implements ReportTemplateResource {
private final PlaceholderService placeholderService;
private final ReportTemplateCache reportTemplateCache;
@Override
@ -30,15 +31,10 @@ public class ReportTemplateController implements ReportTemplateResource {
}
/**
* If a template with the same id is uploaded we evict it from the cache.
*
* @param templateId The id for the uploaded template
*/
@Override
public void uploadTemplate(String templateId) {
public void evictReportTemplateCache(){
TemplateCache.evictCache(templateId);
reportTemplateCache.evictReportTemplateCache();
}
}

View File

@ -0,0 +1,50 @@
package com.iqser.red.service.redaction.report.v1.server.model;
import lombok.AccessLevel;
import lombok.AllArgsConstructor;
import lombok.experimental.FieldDefaults;
@AllArgsConstructor
@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
public class ComponentReportModel {
Integer componentStartColumn;
Integer componentValueIndexColumn;
public boolean isComponentReport() {
return componentStartColumn != null;
}
public int getComponentNameColumn() {
assert isComponentReport();
return componentStartColumn;
}
public int getComponentValueIndexColumn() {
assert writeComponentValueIndices();
return componentValueIndexColumn;
}
public int getComponentValueColumn() {
assert isComponentReport();
if (writeComponentValueIndices()) {
return componentValueIndexColumn + 1;
}
return componentStartColumn + 1;
}
public boolean writeComponentValueIndices() {
return componentValueIndexColumn != null;
}
}

View File

@ -22,14 +22,19 @@ public class ExcelModel {
private Map<Integer, Integer> cellWidths = new HashMap<>();
private Map<CellIdentifier, Cell> cellsToCopyBeforeRedactionPlaceholderRow = new HashMap<>();
private Map<CellIdentifier, Cell> cellsToCopyAfterRedactionPlaceholderRow = new HashMap<>();
private List<Integer> writtenRows = new ArrayList<>();
private boolean rowsBeforeRedactionEntryRowsAdded;
private boolean rssPlaceholdersPresent;
private boolean placeholderInFirstRow;
private boolean firstRowWritten;
private boolean skippedPlaceholderPresent;
private boolean fileAttributesPlaceholderPresent;
private boolean scmFunctionPlaceholderPresent;
private ComponentReportModel componentReport;
private List<FileAttributeModel> fileAttributeColumns;
public boolean isFileAttributesPlaceholderPresent() {
return !fileAttributeColumns.isEmpty();
}
}

View File

@ -9,5 +9,10 @@ public class ImagePlaceholder {
private String placeholder;
private byte[] image;
private ImageType imageType;
public enum ImageType{
PNG, JPEG
}
}

View File

@ -13,10 +13,11 @@ public class ReportRedactionEntry {
private String section;
private String justification;
private String justificationParagraph;
private int paragraphIdx;
private String justificationReason;
private String excerpt;
private String value;
private String entityDisplayName;
private boolean isSkipped;
private String entityClosestHeadline;
}

View File

@ -1,7 +1,6 @@
package com.iqser.red.service.redaction.report.v1.server.service;
package com.iqser.red.service.redaction.report.v1.server.queue;
import static com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration.REPORT_QUEUE;
import static com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration.REPORT_RESULT_QUEUE;
import static com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration.REPORT_RESPONSE_EXCHANGE;
import org.springframework.amqp.AmqpRejectAndDontRequeueException;
import org.springframework.amqp.core.Message;
@ -13,6 +12,8 @@ import org.springframework.stereotype.Service;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.iqser.red.service.redaction.report.v1.api.model.ReportRequestMessage;
import com.iqser.red.service.redaction.report.v1.api.model.ReportResultMessage;
import com.iqser.red.service.redaction.report.v1.server.service.ReportGenerationService;
import com.knecon.fforesight.tenantcommons.TenantContext;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
@ -23,6 +24,8 @@ import lombok.extern.slf4j.Slf4j;
@RequiredArgsConstructor
public class ReportMessageReceiver {
public static final String REPORT_REQUEST_LISTENER_ID = "report-request-listener";
private final ObjectMapper objectMapper;
private final ReportGenerationService reportGenerationService;
private final RabbitTemplate rabbitTemplate;
@ -30,7 +33,7 @@ public class ReportMessageReceiver {
@SneakyThrows
@RabbitHandler
@RabbitListener(queues = REPORT_QUEUE)
@RabbitListener(id = REPORT_REQUEST_LISTENER_ID)
public void receive(Message message) {
ReportRequestMessage reportMessage = objectMapper.readValue(message.getBody(), ReportRequestMessage.class);
@ -53,7 +56,7 @@ public class ReportMessageReceiver {
private void addToReportResultQueue(String userId, String downloadId, String reportFileInformationStorageId) {
rabbitTemplate.convertAndSend(REPORT_RESULT_QUEUE, new ReportResultMessage(userId, downloadId, reportFileInformationStorageId));
rabbitTemplate.convertAndSend(REPORT_RESPONSE_EXCHANGE, TenantContext.getTenantId(), new ReportResultMessage(userId, downloadId, reportFileInformationStorageId));
}
}

View File

@ -0,0 +1,67 @@
package com.iqser.red.service.redaction.report.v1.server.queue;
import static com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration.*;
import java.util.Map;
import java.util.Set;
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Service;
import com.knecon.fforesight.tenantcommons.TenantProvider;
import com.knecon.fforesight.tenantcommons.model.TenantCreatedEvent;
import com.knecon.fforesight.tenantcommons.model.TenantQueueConfiguration;
import com.knecon.fforesight.tenantcommons.model.TenantResponse;
import com.knecon.fforesight.tenantcommons.queue.RabbitQueueFromExchangeService;
import com.knecon.fforesight.tenantcommons.queue.TenantExchangeMessageReceiver;
@Service
public class TenantExchangeMessageReceiverImpl extends TenantExchangeMessageReceiver {
public TenantExchangeMessageReceiverImpl(RabbitQueueFromExchangeService rabbitQueueService, TenantProvider tenantProvider) {
super(rabbitQueueService, tenantProvider);
}
@Override
protected Set<TenantQueueConfiguration> getTenantQueueConfigs() {
return Set.of(TenantQueueConfiguration.builder()
.listenerId(ReportMessageReceiver.REPORT_REQUEST_LISTENER_ID)
.exchangeName(REPORT_REQUEST_EXCHANGE)
.queuePrefix(REPORT_REQUEST_QUEUE_PREFIX)
.dlqName(REPORT_REQUEST_DLQ)
.arguments(Map.of("x-max-priority", 2))
.build());
}
@EventListener(ApplicationReadyEvent.class)
public void onApplicationReady() {
System.out.println("application ready invoked");
super.initializeQueues();
}
@RabbitHandler
@RabbitListener(queues = "#{tenantMessagingConfigurationImpl.getTenantCreatedQueueName()}")
public void reactToTenantCreation(TenantCreatedEvent tenantCreatedEvent) {
super.reactToTenantCreation(tenantCreatedEvent);
}
@RabbitHandler
@RabbitListener(queues = "#{tenantMessagingConfigurationImpl.getTenantDeletedQueueName()}")
public void reactToTenantDeletion(TenantResponse tenantResponse) {
super.reactToTenantDeletion(tenantResponse);
}
}

View File

@ -0,0 +1,36 @@
package com.iqser.red.service.redaction.report.v1.server.service;
import java.util.ArrayList;
import java.util.List;
import org.springframework.stereotype.Service;
@Service
public class CellTextChunkingService {
public static final int MAX_CELL_TEXT_LENGTH = 32767;
public List<String> process(String value) {
List<String> chunks = new ArrayList<>();
if (value == null) {
return chunks;
}
int length = value.length();
if (length < MAX_CELL_TEXT_LENGTH) {
return List.of(value);
}
int startIndex = 0;
while (startIndex < length) {
int endIndex = Math.min(startIndex + MAX_CELL_TEXT_LENGTH, length);
String chunk = value.substring(startIndex, endIndex);
chunks.add(chunk);
startIndex = endIndex;
}
return chunks;
}
}

View File

@ -1,18 +1,16 @@
package com.iqser.red.service.redaction.report.v1.server.service;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.*;
import org.springframework.stereotype.Service;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.componentlog.ComponentLog;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileAttributeConfig;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileModel;
import com.iqser.red.service.redaction.report.v1.server.client.ComponentClient;
import com.iqser.red.service.redaction.report.v1.server.client.FileAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.model.ExcelModel;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.componentlog.ComponentLogEntryValue;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.*;
import com.iqser.red.service.redaction.report.v1.server.client.*;
import com.iqser.red.service.redaction.report.v1.server.model.*;
import lombok.AccessLevel;
import lombok.RequiredArgsConstructor;
@ -25,6 +23,7 @@ public class ComponentRowsReportService {
ComponentClient componentResource;
FileAttributesConfigClient fileAttributesClient;
CellTextChunkingService cellTextChunkingService;
@SuppressWarnings("checkstyle:all")
static int COMPONENT_NAME_COL = 0;
@ -33,36 +32,63 @@ public class ComponentRowsReportService {
public void addComponentRows(Sheet sheet, FileModel fileModel, ExcelModel excelModel) {
ComponentLog componentLog = componentResource.getComponentLog(fileModel.getDossierId(), fileModel.getId(), true);
ComponentLog componentLog = componentResource.getComponentLog(fileModel.getDossierId(), fileModel.getId());
AtomicInteger rowIndex = new AtomicInteger(excelModel.getRedactionPlaceholderRow());
String oecd = getOecdNumber(fileModel);
componentLog.getComponentLogEntries().forEach(componentLogEntry -> {
componentLog.getComponentLogEntries()
.forEach(componentLogEntry -> {
if (componentLogEntry.getComponentValues().isEmpty()) {
return;
}
int componentRowIdx = rowIndex.getAndIncrement();
Row componentRow = sheet.createRow(componentRowIdx);
excelModel.getWrittenRows().add(componentRowIdx);
if (componentLogEntry.getValues().isEmpty()) {
return;
}
Cell componentNameCell = componentRow.createCell(COMPONENT_NAME_COL);
componentNameCell.setCellValue(oecd + "-" + componentLogEntry.getName().replaceAll("_", " "));
String componentName = oecd + "-" + componentLogEntry.getName().replaceAll("_", " ");
for (int valueIdx = 0; valueIdx < componentLogEntry.getComponentValues().size(); valueIdx++) {
String value = componentLogEntry.getComponentValues().get(valueIdx).getValue();
componentRow.createCell(COMPONENT_VALUE_STARTING_COL + valueIdx).setCellValue(value);
}
// Collect chunks for all values
List<List<String>> valueChunksList = new ArrayList<>();
int maxChunks = 0;
});
for (ComponentLogEntryValue valueEntry : componentLogEntry.getValues()) {
String value = valueEntry.getValue();
List<String> chunks = cellTextChunkingService.process(value);
valueChunksList.add(chunks);
if (chunks.size() > maxChunks) {
maxChunks = chunks.size();
}
}
// Create rows for each chunk index
for (int chunkIdx = 0; chunkIdx < maxChunks; chunkIdx++) {
int componentRowIdx = rowIndex.getAndIncrement();
Row componentRow = sheet.createRow(componentRowIdx);
excelModel.getWrittenRows().add(componentRowIdx);
// Component Name Cell
Cell componentNameCell = componentRow.createCell(COMPONENT_NAME_COL);
componentNameCell.setCellValue(componentName);
// Cells for each value
for (int valueIdx = 0; valueIdx < valueChunksList.size(); valueIdx++) {
List<String> chunks = valueChunksList.get(valueIdx);
String chunk = chunkIdx < chunks.size() ? chunks.get(chunkIdx) : "";
Cell valueCell = componentRow.createCell(COMPONENT_VALUE_STARTING_COL + valueIdx);
valueCell.setCellValue(chunk);
CellStyle cellStyle = sheet.getWorkbook().createCellStyle();
cellStyle.setWrapText(true);
valueCell.setCellStyle(cellStyle);
}
}
});
sheet.createRow(rowIndex.get());
excelModel.getWrittenRows().add(rowIndex.get());
excelModel.setRedactionPlaceholderRow(rowIndex.getAndIncrement());
}
@ -73,7 +99,8 @@ public class ComponentRowsReportService {
.filter(f -> f.getLabel().equals("OECD Number"))
.map(FileAttributeConfig::getId)
.findFirst()
.map(oecd -> file.getFileAttributes().get(oecd))
.map(oecd -> file.getFileAttributes()
.get(oecd))
.orElse(null);
}

View File

@ -6,6 +6,7 @@ import java.util.HashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@ -70,8 +71,8 @@ public class EntityLogConverterService {
private Map<String, String> fillMapOfTypeAndEntityDisplayName(String dossierId) {
List<Type> typeList = new ArrayList<>();
typeList.addAll(dictionaryClient.getAllTypesForDossier(dossierId, false));
typeList.addAll(dictionaryClient.getAllTypesForDossierTemplate(dossierClient.getDossierById(dossierId, true, false).getDossierTemplateId(), false));
typeList.addAll(dictionaryClient.getAllTypesForDossier(dossierId, null, true));
typeList.addAll(dictionaryClient.getAllTypesForDossierTemplate(dossierClient.getDossierById(dossierId, true, false).getDossierTemplateId(), null, true));
Map<String, String> mapOfEntityDisplayName = new HashMap<>();
@ -95,11 +96,15 @@ public class EntityLogConverterService {
entityLog.getEntityLogEntry()
.forEach(entry -> {
if (!entry.getState().equals(EntryState.APPLIED)) {
if (entry.getState().equals(EntryState.REMOVED)) {
return;
}
if (entry.getEntryType() == EntryType.HINT || entry.getEntryType() == EntryType.IMAGE_HINT) {
if (entry.getEntryType() == EntryType.HINT) {
return;
}
if (entry.getState() == EntryState.PENDING) {
return;
}
@ -115,6 +120,14 @@ public class EntityLogConverterService {
return;
}
if (entry.getEntryType() == EntryType.IMAGE_HINT && entry.getState().equals(EntryState.SKIPPED)) {
return;
}
if (entry.getEntryType() == EntryType.AREA && (entry.getState().equals(EntryState.SKIPPED) || entry.getState().equals(EntryState.IGNORED))) {
return;
}
if (settings.getMaxRedactionEntryValueLength() > 0 && entry.getValue() != null && entry.getValue().length() > settings.getMaxRedactionEntryValueLength()) {
entry.setValue(entry.getValue().substring(0, settings.getMaxRedactionEntryValueLength()) + "...");
log.info("Truncated value in dossier {}, file {} on pages {} to {} chars",
@ -144,25 +157,27 @@ public class EntityLogConverterService {
if (pages.isEmpty() || !pages.contains(position.getPageNumber())) {
pages.add(position.getPageNumber());
Optional<EntityLogLegalBasis> optionalEntityLogLegalBasis = legalBasisMappings.stream()
.filter(lbm -> lbm.getTechnicalName().equalsIgnoreCase(entry.getLegalBasis()))
.findAny();
EntityLogLegalBasis entityLogLegalBasis = optionalEntityLogLegalBasis.orElse(new EntityLogLegalBasis("", "", "", ""));
reportEntries.add(new ReportRedactionEntry(position.getPageNumber(),
position.x(),
position.y(),
getSection(entry, position),
checkTextForNull(entry.getLegalBasis()) + " " + legalBasisMappings.stream()
.filter(lbm -> lbm.getReason().equalsIgnoreCase(entry.getLegalBasis()))
.findAny()
.map(EntityLogLegalBasis::getDescription)
.orElse(""),
entry.getLegalBasis(),
legalBasisMappings.stream()
.filter(lbm -> lbm.getReason().equalsIgnoreCase(entry.getLegalBasis()))
.findAny()
.map(EntityLogLegalBasis::getDescription)
.orElse(""),
entityLogLegalBasis.getReason() + " " + entityLogLegalBasis.getDescription(),
entityLogLegalBasis.getReason(),
entry.getParagraphPageIdx(),
entityLogLegalBasis.getDescription(),
checkTextForNull(entry.getTextBefore()) + entry.getValue() + checkTextForNull(entry.getTextAfter()),
entry.getValue(),
mapOfEntityDisplayName.get(entry.getType()),
entry.getState() == EntryState.SKIPPED || entry.getState() == EntryState.IGNORED));
mapOfEntityDisplayName.get(entry.getType())
== null ? entry.getType() : mapOfEntityDisplayName.get(entry.getType()),
entry.getState() == EntryState.SKIPPED || entry.getState() == EntryState.IGNORED,
entry.getClosestHeadline()));
}
}
});

View File

@ -0,0 +1,269 @@
package com.iqser.red.service.redaction.report.v1.server.service;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.COMPONENT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.COMPONENT_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.EXCERPT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_ATTRIBUTES_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_ATTRIBUTE_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.INDEX_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_PARAGRAPH_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_REASON_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_TEXT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PAGE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PARAGRAPH_INDEX_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PARAGRAPH_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_VALUE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SCM_FUNCTION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SKIPPED_PLACEHOLDER;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.HorizontalAlignment;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.springframework.stereotype.Service;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileAttributeConfig;
import com.iqser.red.service.redaction.report.v1.server.client.FileAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.model.CellIdentifier;
import com.iqser.red.service.redaction.report.v1.server.model.ComponentReportModel;
import com.iqser.red.service.redaction.report.v1.server.model.ExcelModel;
import com.iqser.red.service.redaction.report.v1.server.model.FileAttributeModel;
import com.iqser.red.service.redaction.report.v1.server.model.PlaceholderInput;
import io.micrometer.core.annotation.Timed;
import lombok.AccessLevel;
import lombok.RequiredArgsConstructor;
import lombok.experimental.FieldDefaults;
import lombok.extern.slf4j.Slf4j;
@Slf4j
@Service
@RequiredArgsConstructor
@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
public class ExcelModelFactory {
private static final Set<String> prefixRedactionPlaceholders = Set.of(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE, FILE_ATTRIBUTE_PLACEHOLDER_BASE);
private static final Set<String> redactionPlaceholders = Set.of(FILE_NAME_PLACEHOLDER,
PAGE_PLACEHOLDER,
PARAGRAPH_PLACEHOLDER,
PARAGRAPH_INDEX_PLACEHOLDER,
JUSTIFICATION_PLACEHOLDER,
EXCERPT_PLACEHOLDER,
JUSTIFICATION_PARAGRAPH_PLACEHOLDER,
JUSTIFICATION_REASON_PLACEHOLDER,
REDACTION_VALUE_PLACEHOLDER,
JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER,
JUSTIFICATION_TEXT_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER,
REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER,
SCM_FUNCTION_PLACEHOLDER,
SKIPPED_PLACEHOLDER,
FILE_ATTRIBUTES_PLACEHOLDER,
INDEX_PLACEHOLDER,
COMPONENT_PLACEHOLDER);
FileAttributesConfigClient fileAttributesConfigClient;
@Timed("redactmanager_calculateExcelModel")
@SuppressWarnings("checkstyle:all")
public ExcelModel calculateExcelModel(Sheet sheet, String dossierTemplateId) {
long start = System.currentTimeMillis();
Map<Integer, Function<PlaceholderInput, String>> placeholderCellPos = new HashMap<>();
Map<Integer, Integer> columnWidths = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyBeforePlaceholderRow = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyAfterPlaceholderRow = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyInPlaceholderRow = new HashMap<>();
Integer componentColumn = null;
Integer componentValueIndexColumn = null;
List<FileAttributeModel> fileAttributeCols = new ArrayList<>();
boolean hasRssPlaceHolders = false;
boolean hasSkippedPlaceholder = false;
int placeholderRow = -1;
boolean placeholderInFirstRow = false;
boolean hasScmFunctionPlaceholder = false;
for (int row = 0; row < sheet.getLastRowNum() + 1; row++) {
Row actualRow = sheet.getRow(row);
if (actualRow == null) {
continue;
}
int lastCellIndex = actualRow.getLastCellNum();
for (int col = 0; col < lastCellIndex; col++) {
Cell cell = actualRow.getCell(col);
if (cell == null) {
continue;
}
boolean isFileAttributesPlaceholder = false;
int columnWidth = sheet.getColumnWidth(col);
columnWidths.put(col, columnWidth);
String cellStringValue = cell.getStringCellValue();
if (row == 0 && cellStringValue.contains("{{")) {
placeholderInFirstRow = true;
}
if (cellStringValue.contains(COMPONENT_PLACEHOLDER_BASE)) {
hasRssPlaceHolders = true;
}
if (cellStringValue.contains(COMPONENT_PLACEHOLDER)) {
componentColumn = col;
}
if (cellStringValue.contains(INDEX_PLACEHOLDER)) {
componentValueIndexColumn = col;
}
if (cellStringValue.contains(SCM_FUNCTION_PLACEHOLDER)) {
hasScmFunctionPlaceholder = true;
}
if (cellStringValue.contains(FILE_ATTRIBUTES_PLACEHOLDER)) {
// For each file attribute we have to replace the column header with the corresponding file attribute.
List<FileAttributeConfig> fileAttributeConfigs = fileAttributesConfigClient.getFileAttributeConfigs(dossierTemplateId);
var iterator = fileAttributeConfigs.iterator();
while (iterator.hasNext()) {
cell = getOrCreateCell(actualRow, col);
FileAttributeConfig fileAttributeConfig = iterator.next();
cell.setCellValue(fileAttributeConfig.getLabel());
cellsToCopyBeforePlaceholderRow.put(new CellIdentifier(row, col), cell);
fileAttributeCols.add(FileAttributeModel.builder().id(fileAttributeConfig.getId()).cell(cell).build());
// If there is more than one file attribute we have to insert a new column, so we shift all columns to the right by 1
// and increase the current and last cell index.
if (iterator.hasNext()) {
lastCellIndex += 1;
col += 1;
sheet.shiftColumns(col, lastCellIndex, 1);
actualRow = sheet.getRow(row);
}
}
isFileAttributesPlaceholder = true;
}
if (!isFileAttributesPlaceholder) {
if (isRedactionPlaceholder(cellStringValue)) {
if (cellStringValue.equals(SKIPPED_PLACEHOLDER)) {
hasSkippedPlaceholder = true;
}
placeholderCellPos.put(col, getFunctionForPlaceHolder(cellStringValue));
placeholderRow = row;
cellsToCopyInPlaceholderRow.put(new CellIdentifier(row, col), cell);
} else {
if (placeholderRow == -1) {
cellsToCopyBeforePlaceholderRow.put(new CellIdentifier(row, col), cell);
} else {
cellsToCopyAfterPlaceholderRow.put(new CellIdentifier(row, col), cell);
}
}
}
}
}
if (hasRssPlaceHolders) {
cellsToCopyBeforePlaceholderRow.putAll(cellsToCopyInPlaceholderRow);
cellsToCopyBeforePlaceholderRow.putAll(cellsToCopyAfterPlaceholderRow);
cellsToCopyAfterPlaceholderRow = new HashMap<>();
}
log.debug("Calculate Placeholder Cells took: {}", System.currentTimeMillis() - start);
ComponentReportModel componentReport = new ComponentReportModel(componentColumn, componentValueIndexColumn);
return new ExcelModel(placeholderCellPos,
placeholderRow,
columnWidths,
cellsToCopyBeforePlaceholderRow,
cellsToCopyAfterPlaceholderRow,
new ArrayList<>(),
false,
hasRssPlaceHolders,
placeholderInFirstRow,
false,
hasSkippedPlaceholder,
hasScmFunctionPlaceholder,
componentReport,
fileAttributeCols);
}
private static Cell getOrCreateCell(Row actualRow, int i) {
Cell cell;
cell = actualRow.getCell(i);
// If there is more than one file attribute, starting from the 2nd one we won't find a cell in the given index so we have to create it.
if (cell == null) {
cell = actualRow.createCell(i);
CellStyle cellStyle = actualRow.getCell(i - 1).getCellStyle();
cellStyle.setAlignment(HorizontalAlignment.CENTER);
cell.setCellStyle(cellStyle);
}
return cell;
}
private boolean isRedactionPlaceholder(String text) {
return prefixRedactionPlaceholders.stream()
.anyMatch(text::startsWith) || redactionPlaceholders.stream()
.anyMatch(text::contains);
}
private Function<PlaceholderInput, String> getFunctionForPlaceHolder(String placeholder) {
if (placeholder.startsWith(FILE_ATTRIBUTE_PLACEHOLDER_BASE)) {
return placeholderInput -> placeholderInput.getPlaceholderModel().getFileAttributeValueByPlaceholder().get(placeholder);
}
if (placeholder.startsWith(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE)) {
return placeholderInput -> placeholderInput.getPlaceholderModel().getDossierAttributesValueByPlaceholder().get(placeholder);
}
return switch (placeholder) {
case FILE_NAME_PLACEHOLDER -> PlaceholderInput::getFilename;
case PAGE_PLACEHOLDER -> input -> String.valueOf(input.getEntry().getPage());
case PARAGRAPH_PLACEHOLDER -> input -> input.getEntry().getSection();
case PARAGRAPH_INDEX_PLACEHOLDER -> input -> String.valueOf(input.getEntry().getParagraphIdx());
case JUSTIFICATION_PLACEHOLDER -> input -> input.getEntry().getJustification();
case JUSTIFICATION_PARAGRAPH_PLACEHOLDER, JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER -> input -> input.getEntry().getJustificationParagraph();
case JUSTIFICATION_REASON_PLACEHOLDER, JUSTIFICATION_TEXT_PLACEHOLDER -> input -> input.getEntry().getJustificationReason();
case EXCERPT_PLACEHOLDER -> input -> input.getEntry().getExcerpt();
case REDACTION_VALUE_PLACEHOLDER ->
input -> input.getEntry().getValue() != null ? input.getEntry().getValue().replaceAll("\n", " ").replaceAll(" {2,}", " ") : input.getEntry()
.getEntityDisplayName();
case REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER -> input -> input.getEntry().getEntityDisplayName();
case SKIPPED_PLACEHOLDER -> input -> input.getEntry().isSkipped() ? "true" : "false";
default -> input -> "";
};
}
}

View File

@ -1,11 +1,6 @@
package com.iqser.red.service.redaction.report.v1.server.service;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.COMPONENT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.DOSSIER_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.EXCERPT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_ATTRIBUTES_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_ATTRIBUTE_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FILE_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_DATE_ENG;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_DATE_ENG_PLACEHOLDER;
@ -15,28 +10,10 @@ import static com.iqser.red.service.redaction.report.v1.server.service.Placehold
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_DATE_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_TIME_ISO;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_TIME_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.INDEX_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_PARAGRAPH_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_REASON_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_TEXT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PAGE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PARAGRAPH_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_VALUE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.RSS_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SCM_FUNCTION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SKIPPED_PLACEHOLDER;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.time.OffsetDateTime;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
@ -53,17 +30,14 @@ import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.ClientAnchor;
import org.apache.poi.ss.usermodel.CreationHelper;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.ss.usermodel.Drawing;
import org.apache.poi.ss.usermodel.HorizontalAlignment;
import org.apache.poi.ss.usermodel.Picture;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.springframework.stereotype.Service;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileAttributeConfig;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileModel;
import com.iqser.red.service.redaction.report.v1.server.client.FileAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.model.CellIdentifier;
import com.iqser.red.service.redaction.report.v1.server.model.ExcelModel;
import com.iqser.red.service.redaction.report.v1.server.model.ImagePlaceholder;
@ -83,30 +57,10 @@ import lombok.extern.slf4j.Slf4j;
@RequiredArgsConstructor
public class ExcelReportGenerationService {
private static final Set<String> prefixRedactionPlaceholders = Set.of(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE, FILE_ATTRIBUTE_PLACEHOLDER_BASE);
private static final Set<String> redactionPlaceholders = Set.of(FILE_NAME_PLACEHOLDER,
PAGE_PLACEHOLDER,
PARAGRAPH_PLACEHOLDER,
JUSTIFICATION_PLACEHOLDER,
EXCERPT_PLACEHOLDER,
JUSTIFICATION_PARAGRAPH_PLACEHOLDER,
JUSTIFICATION_REASON_PLACEHOLDER,
REDACTION_VALUE_PLACEHOLDER,
JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER,
JUSTIFICATION_TEXT_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER,
REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER,
SCM_FUNCTION_PLACEHOLDER,
SKIPPED_PLACEHOLDER,
FILE_ATTRIBUTES_PLACEHOLDER,
INDEX_PLACEHOLDER,
COMPONENT_PLACEHOLDER);
private final ScmReportService componentReportService;
private final FileAttributesConfigClient fileAttributesConfigClient;
private final ComponentRowsReportService componentRowsReportService;
private CreationHelper creationHelper;
final DataFormatter formatter = new DataFormatter();
@Timed("redactmanager_generateExcelReport")
@ -125,70 +79,72 @@ public class ExcelReportGenerationService {
.filter(entry -> !entry.isSkipped())
.collect(Collectors.toList()) : allReportEntries;
try {
for (Sheet sheet : workbook) {
for (Sheet sheet : workbook) {
if (!excelModel.isRowsBeforeRedactionEntryRowsAdded()) {
if (!excelModel.isRowsBeforeRedactionEntryRowsAdded()) {
for (Map.Entry<Integer, Integer> columnWidthEntry : excelModel.getCellWidths().entrySet()) {
sheet.setColumnWidth(columnWidthEntry.getKey(), columnWidthEntry.getValue());
}
addRows(workbook,
sheet,
excelModel.getCellsToCopyBeforeRedactionPlaceholderRow(),
excelModel.isRssPlaceholdersPresent() ? excelModel.getWrittenRows().size() : 0,
placeholderModel,
dossierName,
fileModel.getFilename(),
excelModel);
if (!excelModel.isRssPlaceholdersPresent()) {
excelModel.setRowsBeforeRedactionEntryRowsAdded(true);
}
for (Map.Entry<Integer, Integer> columnWidthEntry : excelModel.getCellWidths().entrySet()) {
sheet.setColumnWidth(columnWidthEntry.getKey(), columnWidthEntry.getValue());
}
if (!excelModel.isRssPlaceholdersPresent() && !excelModel.isFileAttributesPlaceholderPresent() && !excelModel.isScmFunctionPlaceholderPresent()) {
addRedactionEntryRows(sheet, reportEntries, fileModel.getFilename(), excelModel, placeholderModel);
}
addRows(workbook,
sheet,
excelModel.getCellsToCopyBeforeRedactionPlaceholderRow(),
excelModel.isRssPlaceholdersPresent() ? excelModel.getWrittenRows().size() : 0,
placeholderModel,
dossierName,
fileModel.getFilename(),
excelModel);
if (excelModel.isFileAttributesPlaceholderPresent()) {
componentReportService.addScmRows(sheet, fileModel, excelModel);
if (!excelModel.isRssPlaceholdersPresent()) {
excelModel.setRowsBeforeRedactionEntryRowsAdded(true);
}
}
if (isRedactionReport(excelModel)) {
addRedactionEntryRows(sheet, reportEntries, fileModel.getFilename(), excelModel, placeholderModel);
} else {
if (excelModel.getComponentReport().isComponentReport()) {
componentReportService.addComponentRows(sheet, fileModel, excelModel);
}
if (excelModel.isScmFunctionPlaceholderPresent()) {
componentRowsReportService.addComponentRows(sheet, fileModel, excelModel);
}
if (isLastFile) {
addRows(workbook,
sheet,
excelModel.getCellsToCopyAfterRedactionPlaceholderRow(),
excelModel.getWrittenRows().size(),
placeholderModel,
dossierName,
fileModel.getFilename(),
excelModel);
}
}
log.info("Report Generation took: {} for file with id {}, pageCount: {}, entityLogEntryCount: {}, reportName: {}, className: {}",
System.currentTimeMillis() - start,
fileModel.getId(),
fileModel.getNumberOfPages(),
reportEntries.size(),
reportTemplateName,
getClass().getSimpleName());
} catch (Exception e) {
throw new RuntimeException(e);
if (isLastFile) {
addRows(workbook,
sheet,
excelModel.getCellsToCopyAfterRedactionPlaceholderRow(),
excelModel.getWrittenRows().size(),
placeholderModel,
dossierName,
fileModel.getFilename(),
excelModel);
}
}
log.info("Report Generation took: {} for file with id {}, pageCount: {}, entityLogEntryCount: {}, reportName: {}, className: {}",
System.currentTimeMillis() - start,
fileModel.getId(),
fileModel.getNumberOfPages(),
reportEntries.size(),
reportTemplateName,
getClass().getSimpleName());
}
private static boolean isRedactionReport(ExcelModel excelModel) {
return !(excelModel.isRssPlaceholdersPresent()
|| excelModel.isFileAttributesPlaceholderPresent()
|| excelModel.isScmFunctionPlaceholderPresent()
|| excelModel.getComponentReport().isComponentReport());
}
@SneakyThrows
private void addRows(SXSSFWorkbook workbook,
Sheet sheet,
Map<CellIdentifier, Cell> copiedCells,
@ -207,6 +163,11 @@ public class ExcelReportGenerationService {
continue;
}
log.debug("Copying cell {},{} with content {} from template",
cellsToCopyEntry.getKey().getRowIndex(),
cellsToCopyEntry.getKey().getColumnIndex(),
formatter.formatCellValue(cellsToCopyEntry.getValue()));
int indexToAddRow = cellsToCopyEntry.getKey().getRowIndex() + numberOfRowsToShift - skipRows;
if (!createdCopyRows.contains(indexToAddRow)) {
@ -240,11 +201,18 @@ public class ExcelReportGenerationService {
Map<Integer, Function<PlaceholderInput, String>> placeholderCellPos = excelModel.getPlaceholderCellPos();
reportEntries.forEach(entry -> {
PlaceholderInput placeholderInput = new PlaceholderInput(filename, entry, placeholderModel, null);
sheet.createRow(rowIndex.get());
excelModel.getWrittenRows().add(rowIndex.get());
for (Map.Entry<Integer, Function<PlaceholderInput, String>> entry1 : placeholderCellPos.entrySet()) {
Cell cell = sheet.getRow(rowIndex.get()).createCell(entry1.getKey());
cell.setCellValue(entry1.getValue().apply(new PlaceholderInput(filename, entry, placeholderModel, null)) == null ? "" : entry1.getValue().apply(new PlaceholderInput(filename, entry, placeholderModel, null)));
for (Map.Entry<Integer, Function<PlaceholderInput, String>> colIdxToCellValueTransform : placeholderCellPos.entrySet()) {
Cell cell = sheet.getRow(rowIndex.get()).createCell(colIdxToCellValueTransform.getKey());
String cellValue = colIdxToCellValueTransform.getValue().apply(placeholderInput);
if (cellValue == null) {
cellValue = "";
}
cell.setCellValue(cellValue);
}
rowIndex.getAndIncrement();
});
@ -255,9 +223,6 @@ public class ExcelReportGenerationService {
}
private void replacePlaceholders(Cell cell, PlaceholderModel placeholderModel, String dossierName, String filename) {
for (String placeholder : placeholderModel.getPlaceholders()) {
@ -269,7 +234,8 @@ public class ExcelReportGenerationService {
}
private void replacePlaceholdersForImagePlaceholder(SXSSFWorkbook workbook, Sheet sheet, Cell cell, PlaceholderModel placeholderModel) throws IOException {
@SneakyThrows
private void replacePlaceholdersForImagePlaceholder(SXSSFWorkbook workbook, Sheet sheet, Cell cell, PlaceholderModel placeholderModel) {
for (ImagePlaceholder imagePlaceholder : placeholderModel.getImagePlaceholders()) {
@ -282,9 +248,9 @@ public class ExcelReportGenerationService {
PixelUtil.heightUnits2Pixel(cell.getRow().getHeight()));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(image, "jpg", baos);
ImageIO.write(image, imagePlaceholder.getImageType().equals(ImagePlaceholder.ImageType.PNG) ? "png" : "jpg", baos);
baos.flush();
int pictureIdx = workbook.addPicture(baos.toByteArray(), SXSSFWorkbook.PICTURE_TYPE_JPEG);
int pictureIdx = workbook.addPicture(baos.toByteArray(), imagePlaceholder.getImageType().equals(ImagePlaceholder.ImageType.PNG) ? SXSSFWorkbook.PICTURE_TYPE_PNG : SXSSFWorkbook.PICTURE_TYPE_JPEG);
baos.close();
CreationHelper helper = getCreationHelper(workbook);
@ -381,162 +347,6 @@ public class ExcelReportGenerationService {
}
@Timed("redactmanager_calculateExcelModel")
@SuppressWarnings("checkstyle:all")
public ExcelModel calculateExcelModel(Sheet sheet, String dossierTemplateId) {
long start = System.currentTimeMillis();
Map<Integer, Function<PlaceholderInput, String>> placeholderCellPos = new HashMap<>();
Map<Integer, Integer> columnWidths = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyBeforePlaceholderRow = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyAfterPlaceholderRow = new HashMap<>();
Map<CellIdentifier, Cell> cellsToCopyInPlaceholderRow = new HashMap<>();
boolean hasRssPlaceHolders = false;
boolean hasSkippedPlaceholder = false;
int placeholderRow = -1;
boolean placeholderInFirstRow = false;
boolean fileAttributesPlaceholder = false;
boolean hasScmFunctionPlaceholder = false;
for (int j = 0; j < sheet.getLastRowNum() + 1; j++) {
Row actualRow = sheet.getRow(j);
if (actualRow != null) {
int lastCellIndex = actualRow.getLastCellNum();
for (int i = 0; i < lastCellIndex; i++) {
Cell cell = sheet.getRow(j).getCell(i);
if (cell != null) {
boolean skipCopy = false;
int columnWidth = sheet.getColumnWidth(i);
columnWidths.put(i, columnWidth);
String cellStringValue = cell.getStringCellValue();
if (j == 0 && cellStringValue.contains("{{")) {
placeholderInFirstRow = true;
}
if (cellStringValue.contains(RSS_PLACEHOLDER_BASE)) {
hasRssPlaceHolders = true;
}
if (cellStringValue.contains(SCM_FUNCTION_PLACEHOLDER)) {
hasScmFunctionPlaceholder = true;
}
if (cellStringValue.contains(FILE_ATTRIBUTES_PLACEHOLDER)) {
fileAttributesPlaceholder = true;
// For each file attribute we have to replace the column header with the corresponding file attribute.
List<FileAttributeConfig> fileAttributeConfigs = fileAttributesConfigClient.getFileAttributeConfigs(dossierTemplateId);
var iterator = fileAttributeConfigs.iterator();
while (iterator.hasNext()) {
cell = sheet.getRow(j).getCell(i);
// If there is more than one file attribute, starting from the 2nd one we won't find a cell in the given index so we have to create it.
if (cell == null) {
cell = sheet.getRow(j).createCell(i);
CellStyle cellStyle = sheet.getRow(j).getCell(i-1).getCellStyle();
cellStyle.setAlignment(HorizontalAlignment.CENTER);
cell.setCellStyle(cellStyle);
}
cell.setCellValue(iterator.next().getLabel());
cellsToCopyBeforePlaceholderRow.put(new CellIdentifier(j, i), cell);
// If there is more than one file attribute we have to insert a new column, so we shift all columns to the right by 1
// and increase the current and last cell index.
if (iterator.hasNext()) {
lastCellIndex += 1;
i += 1;
sheet.shiftColumns(i, lastCellIndex, 1);
}
}
skipCopy = true;
}
if (!skipCopy) {
if (isRedactionPlaceholder(cellStringValue)) {
if (cellStringValue.equals(SKIPPED_PLACEHOLDER)) {
hasSkippedPlaceholder = true;
}
placeholderCellPos.put(i, getFunctionForPlaceHolder(cellStringValue));
placeholderRow = j;
cellsToCopyInPlaceholderRow.put(new CellIdentifier(j, i), cell);
} else {
if (placeholderRow == -1) {
cellsToCopyBeforePlaceholderRow.put(new CellIdentifier(j, i), cell);
} else {
cellsToCopyAfterPlaceholderRow.put(new CellIdentifier(j, i), cell);
}
}
}
}
}
}
}
if (hasRssPlaceHolders) {
cellsToCopyBeforePlaceholderRow.putAll(cellsToCopyInPlaceholderRow);
cellsToCopyBeforePlaceholderRow.putAll(cellsToCopyAfterPlaceholderRow);
cellsToCopyAfterPlaceholderRow = new HashMap<>();
}
log.debug("Calculate Placeholder Cells took: {}", System.currentTimeMillis() - start);
return new ExcelModel(placeholderCellPos,
placeholderRow,
columnWidths,
cellsToCopyBeforePlaceholderRow,
cellsToCopyAfterPlaceholderRow,
new ArrayList<>(),
false,
hasRssPlaceHolders,
placeholderInFirstRow,
false,
hasSkippedPlaceholder,
fileAttributesPlaceholder,
hasScmFunctionPlaceholder);
}
private boolean isRedactionPlaceholder(String text) {
return prefixRedactionPlaceholders.stream().anyMatch(text::startsWith) || redactionPlaceholders.stream().anyMatch(text::contains);
}
private Function<PlaceholderInput, String> getFunctionForPlaceHolder(String placeholder) {
if (placeholder.startsWith(FILE_ATTRIBUTE_PLACEHOLDER_BASE)) {
return placeholderInput -> placeholderInput.getPlaceholderModel().getFileAttributeValueByPlaceholder().get(placeholder);
}
if (placeholder.startsWith(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE)) {
return placeholderInput -> placeholderInput.getPlaceholderModel().getDossierAttributesValueByPlaceholder().get(placeholder);
}
return switch (placeholder) {
case FILE_NAME_PLACEHOLDER -> PlaceholderInput::getFilename;
case PAGE_PLACEHOLDER -> input -> String.valueOf(input.getEntry().getPage());
case PARAGRAPH_PLACEHOLDER -> input -> input.getEntry().getSection();
case JUSTIFICATION_PLACEHOLDER -> input -> input.getEntry().getJustification();
case JUSTIFICATION_PARAGRAPH_PLACEHOLDER, JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER -> input -> input.getEntry().getJustificationParagraph();
case JUSTIFICATION_REASON_PLACEHOLDER, JUSTIFICATION_TEXT_PLACEHOLDER -> input -> input.getEntry().getJustificationReason();
case EXCERPT_PLACEHOLDER -> input -> input.getEntry().getExcerpt();
case REDACTION_VALUE_PLACEHOLDER ->
input -> input.getEntry().getValue() != null ? input.getEntry().getValue().replaceAll("\n", " ").replaceAll(" {2,}", " ") : input.getEntry()
.getEntityDisplayName();
case REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER -> input -> input.getEntry().getEntityDisplayName();
case SKIPPED_PLACEHOLDER -> input -> input.getEntry().isSkipped() ? "true" : "false";
default -> input -> "";
};
}
@SneakyThrows
public byte[] toByteArray(SXSSFWorkbook workbook) {

View File

@ -15,13 +15,13 @@ import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemp
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileModel;
import com.iqser.red.service.redaction.report.v1.api.model.ReportType;
import com.iqser.red.service.redaction.report.v1.api.model.StoredFileInformation;
import com.iqser.red.service.redaction.report.v1.server.cache.ReportTemplateCache;
import com.iqser.red.service.redaction.report.v1.server.model.ExcelModel;
import com.iqser.red.service.redaction.report.v1.server.model.MultiFileWorkbook;
import com.iqser.red.service.redaction.report.v1.server.model.PlaceholderModel;
import com.iqser.red.service.redaction.report.v1.server.model.ReportRedactionEntry;
import com.iqser.red.service.redaction.report.v1.server.model.ReportTemplatesModel;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageServiceAsyncWrapper;
import com.iqser.red.service.redaction.report.v1.server.utils.TemplateCache;
import lombok.AccessLevel;
import lombok.RequiredArgsConstructor;
@ -33,10 +33,11 @@ import lombok.experimental.FieldDefaults;
@SuppressWarnings("PMD")
public class ExcelReportTemplateService {
ReportStorageService reportStorageService;
ExcelReportGenerationService excelTemplateReportGenerationService;
GeneratePlaceholderService generatePlaceholderService;
ReportStorageServiceAsyncWrapper reportStorageServiceAsyncWrapper;
ReportTemplateCache reportTemplateCache;
ExcelModelFactory excelModelFactory;
public CompletableFuture<StoredFileInformation> createExcelReportFromTemplateAsync(Dossier dossier,
@ -47,14 +48,14 @@ public class ExcelReportTemplateService {
List<ReportRedactionEntry> reportEntries,
ReportTemplate reportTemplate) {
byte[] excelTemplate = TemplateCache.getTemplate(reportTemplate.getStorageId(), reportStorageService);
byte[] excelTemplate = reportTemplateCache.getTemplate(reportTemplate.getStorageId());
try (ByteArrayInputStream is = new ByteArrayInputStream(excelTemplate)) {
XSSFWorkbook readWorkbook = new XSSFWorkbook(is);
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
for (Sheet sheet : readWorkbook) {
writeWorkbook.createSheet(sheet.getSheetName());
}
var excelModel = excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
if (excelModel.isRssPlaceholdersPresent()) {
generatePlaceholderService.resolveRssValues(fileStatus, placeholderModel);
}
@ -80,18 +81,17 @@ public class ExcelReportTemplateService {
public void prepareExcelReportTemplates(String dossierTemplateId, String templateId, ReportTemplate reportTemplate, ReportTemplatesModel reportTemplatesModel) {
byte[] excelTemplate = TemplateCache.getTemplate(reportTemplate.getStorageId(), reportStorageService);
byte[] excelTemplate = reportTemplateCache.getTemplate(reportTemplate.getStorageId());
try (ByteArrayInputStream is = new ByteArrayInputStream(excelTemplate)) {
XSSFWorkbook readWorkbook = new XSSFWorkbook(is);
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
for (Sheet sheet : readWorkbook) {
writeWorkbook.createSheet(sheet.getSheetName());
}
MultiFileWorkbook multiFileWorkbook = new MultiFileWorkbook(readWorkbook,
writeWorkbook,
templateId,
reportTemplate.getFileName(),
excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossierTemplateId));
ExcelModel excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossierTemplateId);
MultiFileWorkbook multiFileWorkbook = new MultiFileWorkbook(readWorkbook, writeWorkbook, templateId, reportTemplate.getFileName(), excelModel);
reportTemplatesModel.multiFileWorkbookReportTemplates.add(multiFileWorkbook);
} catch (IOException e) {
throw new RuntimeException("Could not generate multifile excel report.");

View File

@ -7,7 +7,7 @@ import static com.iqser.red.service.redaction.report.v1.server.service.Placehold
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_DATE_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_TIME_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.IUCLID_FUNCTION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.RSS_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.COMPONENT_PLACEHOLDER_BASE;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER;
@ -43,6 +43,7 @@ public class GeneratePlaceholderService {
private final DossierAttributesClient dossierAttributesClient;
private final DossierAttributesConfigClient dossierAttributesConfigClient;
private final RSSPoc2Service rSSService;
public static String PNG_IMAGE_REGEX = "data:image/png;base64.*";
public PlaceholderModel buildPlaceholders(Dossier dossier) {
@ -63,10 +64,15 @@ public class GeneratePlaceholderService {
if (dossierAttribute.getValue() != null) {
if (dossierAttribute.getValue().startsWith("data:")) {
imagePlaceholders.add(new ImagePlaceholder(attributeConfig.getPlaceholder(),
Base64.getDecoder().decode(dossierAttribute.getValue().split(",")[1])));
Base64.getDecoder().decode(dossierAttribute.getValue().split(",")[1]),
isPng(dossierAttribute.getValue()) ? ImagePlaceholder.ImageType.PNG : ImagePlaceholder.ImageType.JPEG));
} else {
imagePlaceholders.add(new ImagePlaceholder(attributeConfig.getPlaceholder(), Base64.getDecoder().decode(dossierAttribute.getValue())));
imagePlaceholders.add(new ImagePlaceholder(attributeConfig.getPlaceholder(),
Base64.getDecoder().decode(dossierAttribute.getValue()),
ImagePlaceholder.ImageType.JPEG));
}
}
} else {
@ -90,6 +96,12 @@ public class GeneratePlaceholderService {
}
private boolean isPng(String imageString) {
return imageString.startsWith("data:image/png");
}
private Map<String, String> getFileAttributePlaceholders(String dossierTemplateId) {
Map<String, String> fileAttributePlaceholders = new HashMap<>();
@ -103,14 +115,14 @@ public class GeneratePlaceholderService {
public Set<String> getDefaultPlaceholders() {
return new HashSet<>(Set.of(FILE_NAME_PLACEHOLDER,
FORMAT_DATE_ISO_PLACEHOLDER,
FORMAT_DATE_GER_PLACEHOLDER,
FORMAT_DATE_ENG_PLACEHOLDER,
FORMAT_TIME_ISO_PLACEHOLDER,
DOSSIER_NAME_PLACEHOLDER,
IUCLID_FUNCTION_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER));
FORMAT_DATE_ISO_PLACEHOLDER,
FORMAT_DATE_GER_PLACEHOLDER,
FORMAT_DATE_ENG_PLACEHOLDER,
FORMAT_TIME_ISO_PLACEHOLDER,
DOSSIER_NAME_PLACEHOLDER,
IUCLID_FUNCTION_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER));
}
@ -131,7 +143,7 @@ public class GeneratePlaceholderService {
if (!rssResponse.getFiles().isEmpty()) {
for (Map.Entry<String, SCMComponent> rssEntry : rssResponse.getFiles().get(0).getResult().entrySet()) {
rssPlaceholders.put(RSS_PLACEHOLDER_BASE + rssEntry.getKey() + "}}", rssEntry.getValue().getOriginalValue());
rssPlaceholders.put(COMPONENT_PLACEHOLDER_BASE + rssEntry.getKey() + "}}", rssEntry.getValue().getOriginalValue());
}
}

View File

@ -40,6 +40,7 @@ public class PlaceholderService {
public static final String FILE_NAME_PLACEHOLDER = "{{file.name}}";
public static final String PAGE_PLACEHOLDER = "{{redaction.page}}";
public static final String PARAGRAPH_PLACEHOLDER = "{{redaction.paragraph}}";
public static final String PARAGRAPH_INDEX_PLACEHOLDER = "{{redaction.paragraphIdx}}";
public static final String JUSTIFICATION_PLACEHOLDER = "{{redaction.justification}}";
public static final String JUSTIFICATION_PARAGRAPH_PLACEHOLDER = "{{redaction.justificationParagraph}}";
public static final String JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER = "{{redaction.justificationLegalBasis}}";
@ -52,11 +53,13 @@ public class PlaceholderService {
public static final String SCM_FUNCTION_PLACEHOLDER = "{{function.scm}}";
public static final String FILE_ATTRIBUTE_PLACEHOLDER_BASE = "{{file.attribute.";
public static final String DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE = "{{dossier.attribute.";
public static final String RSS_PLACEHOLDER_BASE = "{{component.";
public static final String COMPONENT_PLACEHOLDER_BASE = "{{component.";
public static final String COMPONENT_PLACEHOLDER = "{{component}}";
public static final String FILE_ATTRIBUTES_PLACEHOLDER = "{{file.attributes}}";
public static final String INDEX_PLACEHOLDER = "{{index}}";
public static final String REDACTION_ENTITY_HEADLINE_PLACEHOLDER = "{{redaction.entity.closestHeadline}}";
public static final DateTimeFormatter FORMAT_DATE_ISO = DateTimeFormatter.ofPattern("yyyy-MM-dd");
public static final DateTimeFormatter FORMAT_DATE_GER = DateTimeFormatter.ofPattern("dd.MM.yyyy");
public static final DateTimeFormatter FORMAT_DATE_ENG = DateTimeFormatter.ofPattern("MM/dd/yyyy");
@ -77,6 +80,7 @@ public class PlaceholderService {
public static final Set<String> GENERAL_PLACEHOLDERS = Set.of(FILE_NAME_PLACEHOLDER,
PAGE_PLACEHOLDER,
PARAGRAPH_PLACEHOLDER,
PARAGRAPH_INDEX_PLACEHOLDER,
JUSTIFICATION_PLACEHOLDER,
EXCERPT_PLACEHOLDER,
FORMAT_DATE_ISO_PLACEHOLDER,
@ -91,7 +95,7 @@ public class PlaceholderService {
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER,
REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER,
SKIPPED_PLACEHOLDER);
SKIPPED_PLACEHOLDER, REDACTION_ENTITY_HEADLINE_PLACEHOLDER);
private final ReportTemplateClient reportTemplateClient;
private final ReportStorageService reportStorageService;

View File

@ -7,9 +7,6 @@ import java.util.Collections;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.xwpf.usermodel.XWPFDocument;
import org.springframework.stereotype.Service;
@ -29,12 +26,10 @@ import com.iqser.red.service.redaction.report.v1.server.model.ReportTemplatesMod
import com.iqser.red.service.redaction.report.v1.server.settings.ReportTemplateSettings;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageServiceAsyncWrapper;
import com.iqser.red.service.redaction.report.v1.server.utils.TemplateCache;
import io.micrometer.core.annotation.Timed;
import lombok.AccessLevel;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.experimental.FieldDefaults;
import lombok.extern.slf4j.Slf4j;
@ -57,7 +52,6 @@ public class ReportGenerationService {
ReportTemplateService reportTemplateService;
@SneakyThrows
@Timed("redactmanager_generateReports")
public String generateReports(ReportRequestMessage reportMessage) {
@ -75,9 +69,8 @@ public class ReportGenerationService {
var isLastFile = fileIdIndex == reportMessage.getFileIds().size() - 1;
String dossierId = dossier.getId();
String fileId = reportMessage.getFileIds()
.get(fileIdIndex);
String dossierName = dossier.getDossierName();
String fileId = reportMessage.getFileIds().get(fileIdIndex);
String dossierName = dossier.getName();
var fileStatus = fileStatusClient.getFileStatus(dossierId, fileId);
generatePlaceholderService.resolveFileAttributeValues(fileStatus, placeholderModel);
@ -222,12 +215,12 @@ public class ReportGenerationService {
return CompletableFuture.allOf(singleFilesTemplates.stream()
.map(reportTemplate -> reportTemplateService.createReportFromTemplateAsync(dossier,
fileStatus,
placeholderModel,
reportTemplate.getFileName(),
downloadId,
reportEntries,
reportTemplate).thenAccept(storedFileInformation::add))
fileStatus,
placeholderModel,
reportTemplate.getFileName(),
downloadId,
reportEntries,
reportTemplate).thenAccept(storedFileInformation::add))
.toArray(CompletableFuture[]::new));
}

View File

@ -38,148 +38,123 @@ public class ScmReportService {
final ComponentClient componentResource;
final FileAttributesConfigClient fileAttributesClient;
final CellTextChunkingService cellTextChunkingService;
final DataFormatter formatter = new DataFormatter();
Row row;
public void addScmRows(Sheet sheet, FileModel fileModel, ExcelModel excelModel) {
var firstRow = sheet.getRow(0);
public void addComponentRows(Sheet sheet, FileModel fileModel, ExcelModel excelModel) {
int firstRowIndex = 1;
while (firstRow == null) {
firstRow = sheet.getRow(firstRowIndex);
firstRowIndex++;
}
ComponentLog componentLog = componentResource.getComponentLog(fileModel.getDossierId(), fileModel.getId());
// sometimes first row is identified wrong. this makes sure we always identify the correct first row which contains the
// header cells as we need them for file attributes (e.g. OECD Number)
Cell firstCell = firstRow.getCell(0);
if (formatter.formatCellValue(firstCell).equals("File")) {
row = firstRow;
} else {
firstRow = row;
}
int fileColumnIndex = 0;
int textColumnIndex = firstRow.getLastCellNum() - 1;
int valueIndexColumn = firstRow.getLastCellNum() - 2;
int componentColumnIndex = firstRow.getLastCellNum() - 3;
CellUtil.setAlignment(firstRow.getCell(textColumnIndex), HorizontalAlignment.LEFT);
ComponentLog componentLog = componentResource.getComponentLog(fileModel.getDossierId(), fileModel.getId(), true);
AtomicInteger rowIndex = new AtomicInteger(excelModel.getRedactionPlaceholderRow());
AtomicInteger rowIndexCounter = new AtomicInteger(excelModel.getRedactionPlaceholderRow());
Map<String, Integer> componentIndexMap = new HashMap<>();
Row finalFirstRow = firstRow;
componentLog.getComponentLogEntries()
.forEach(componentLogEntry -> {
// Process each ComponentLogEntryValue
addOtherCells(sheet, excelModel, rowIndexCounter, componentIndexMap, componentLogEntry, fileModel);
});
componentLog.getComponentLogEntries().forEach(componentLogEntry -> {
autosizeColumns(sheet, rowIndexCounter);
if (componentLogEntry.getComponentValues().isEmpty()) {
return;
}
sheet.createRow(rowIndexCounter.get());
excelModel.getWrittenRows().add(rowIndexCounter.get());
excelModel.setRedactionPlaceholderRow(rowIndexCounter.getAndIncrement());
}
int rowIndexAndIncrement = rowIndex.getAndIncrement();
Row row = sheet.createRow(rowIndexAndIncrement);
excelModel.getWrittenRows().add(rowIndexAndIncrement);
// Filename cell
Cell fileNameCell = row.createCell(fileColumnIndex);
fileNameCell.setCellValue(fileModel.getFilename() == null ? "" : fileModel.getFilename());
CellUtil.setVerticalAlignment(fileNameCell, VerticalAlignment.TOP);
// Component, file attributes and index cell
addOtherCells(sheet, excelModel, rowIndex, componentIndexMap, componentLogEntry, row, fileModel, finalFirstRow, componentColumnIndex, textColumnIndex, valueIndexColumn);
});
private static void autosizeColumns(Sheet sheet, AtomicInteger rowIndexCounter) {
// Autosize all cells besides the last column because the extracted text should be multiline
if (sheet instanceof SXSSFSheet) {
((SXSSFSheet) sheet).trackAllColumnsForAutoSizing();
for (int i = 0; i < firstRow.getLastCellNum()-1; i++) {
sheet.autoSizeColumn(i);
Row row = sheet.getRow(rowIndexCounter.get() - 1);
if (row != null) {
for (int i = 0; i < row.getLastCellNum(); i++) {
sheet.autoSizeColumn(i);
}
}
}
sheet.createRow(rowIndex.get());
excelModel.getWrittenRows().add(rowIndex.get());
excelModel.setRedactionPlaceholderRow(rowIndex.getAndIncrement());
}
private void addOtherCells(Sheet sheet,
ExcelModel excelModel,
AtomicInteger rowIndex,
AtomicInteger rowIndexCounter,
Map<String, Integer> componentIndexMap,
ComponentLogEntry componentLogEntry,
Row row,
FileModel fileModel,
Row firstRow,
int componentColumnIndex,
int textColumnIndex,
int valueIndexColumn) {
FileModel fileModel) {
int rowIndexAndIncrement;
Cell componentNameCell = row.createCell(componentColumnIndex);
String componentValue = componentLogEntry.getName().replaceAll("_", " ");
componentNameCell.setCellValue(componentValue);
List<FileAttributeModel> fileAttributeModels = buildFileAttributeModels(fileModel, firstRow);
// FileAttributes cells
addFileAttribute(row, fileModel, fileAttributeModels);
List<FileAttributeModel> fileAttributeModels = excelModel.getFileAttributeColumns();
if (componentLogEntry.getValues().isEmpty()) {
writeRow(sheet, excelModel, rowIndexCounter, fileModel, "", componentValue, componentIndexMap, fileAttributeModels);
return;
}
for (ComponentLogEntryValue componentLogEntryValue : componentLogEntry.getValues()) {
var iterator = componentLogEntry.getComponentValues().iterator();
while (iterator.hasNext()) {
ComponentLogEntryValue componentLogEntryValue = iterator.next();
Cell indexCell = row.createCell(valueIndexColumn);
Cell textExtractionCell = row.createCell(textColumnIndex);
List<String> textChunks = cellTextChunkingService.process(componentLogEntryValue.getValue());
addTextExtraction(componentLogEntryValue, textExtractionCell);
if (componentIndexMap.containsKey(componentValue)) {
int indexValue = componentIndexMap.get(componentValue);
indexCell.setCellValue(indexValue + 1);
Cell componentCell = row.createCell(componentColumnIndex);
componentCell.setCellValue(componentValue);
Cell fileNameCell = row.createCell(0);
fileNameCell.setCellValue(fileModel.getFilename());
addFileAttribute(row, fileModel, fileAttributeModels);
componentIndexMap.put(componentValue, indexValue + 1);
} else {
componentIndexMap.put(componentValue, 1);
indexCell.setCellValue(1);
}
CellUtil.setAlignment(indexCell, HorizontalAlignment.CENTER);
CellUtil.setVerticalAlignment(componentNameCell, VerticalAlignment.TOP);
CellUtil.setVerticalAlignment(indexCell, VerticalAlignment.TOP);
if (iterator.hasNext()) {
rowIndexAndIncrement = rowIndex.getAndIncrement();
row = sheet.createRow(rowIndexAndIncrement);
excelModel.getWrittenRows().add(rowIndexAndIncrement);
for (String chunk : textChunks) {
writeRow(sheet, excelModel, rowIndexCounter, fileModel, chunk, componentValue, componentIndexMap, fileAttributeModels);
}
}
}
private void addTextExtraction(ComponentLogEntryValue componentLogEntryValue, Cell textExtractionCell) {
private void writeRow(Sheet sheet,
ExcelModel excelModel,
AtomicInteger rowIndexCounter,
FileModel fileModel,
String chunk,
String componentValue,
Map<String, Integer> componentIndexMap,
List<FileAttributeModel> fileAttributeModels) {
textExtractionCell.setCellValue(componentLogEntryValue.getValue() == null ? "" : componentLogEntryValue.getValue());
CellStyle cellStyle = textExtractionCell.getCellStyle();
int componentIndex = incrementAndGet(componentValue, componentIndexMap);
int currentRowIdx = rowIndexCounter.getAndIncrement();
Row row = sheet.createRow(currentRowIdx);
excelModel.getWrittenRows().add(currentRowIdx);
Cell fileNameCell = row.createCell(0);
fileNameCell.setCellValue(fileModel.getFilename() == null ? "" : fileModel.getFilename());
CellUtil.setVerticalAlignment(fileNameCell, VerticalAlignment.TOP);
Cell componentNameCell = row.createCell(excelModel.getComponentReport().getComponentNameColumn());
componentNameCell.setCellValue(componentValue);
if (excelModel.getComponentReport().writeComponentValueIndices()) {
Cell indexCell = row.createCell(excelModel.getComponentReport().getComponentValueIndexColumn());
indexCell.setCellValue(componentIndex);
CellUtil.setAlignment(indexCell, HorizontalAlignment.CENTER);
CellUtil.setVerticalAlignment(indexCell, VerticalAlignment.TOP);
}
Cell textCell = row.createCell(excelModel.getComponentReport().getComponentValueColumn());
textCell.setCellValue(chunk);
CellStyle cellStyle = sheet.getWorkbook().createCellStyle();
cellStyle.setWrapText(true);
textExtractionCell.setCellStyle(cellStyle);
textCell.setCellStyle(cellStyle);
addFileAttribute(row, fileModel, fileAttributeModels);
}
private static int incrementAndGet(String componentValue, Map<String, Integer> componentIndexMap) {
int componentIndex = componentIndexMap.getOrDefault(componentValue, 0);
componentIndex++;
componentIndexMap.put(componentValue, componentIndex);
return componentIndex;
}
private static int increment(Map<String, Integer> componentIndexMap, int componentIndex, String componentValue) {
return componentIndex;
}
private void addFileAttribute(Row row, FileModel fileModel, List<FileAttributeModel> fileAttributeModels) {
for (FileAttributeModel fileAttributeModel : fileAttributeModels) {
@ -202,6 +177,7 @@ public class ScmReportService {
return fileAttributes;
}
private List<FileAttributeModel> buildFileAttributeModels(FileModel file, Row firstRow) {
List<FileAttributeModel> fileAttributeModels = new ArrayList<>();
@ -217,5 +193,4 @@ public class ScmReportService {
return fileAttributeModels;
}
}

View File

@ -20,6 +20,7 @@ import static com.iqser.red.service.redaction.report.v1.server.service.Placehold
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_REASON_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.JUSTIFICATION_TEXT_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PAGE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PARAGRAPH_INDEX_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.PARAGRAPH_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_VALUE_PLACEHOLDER;
@ -81,6 +82,7 @@ import lombok.extern.slf4j.Slf4j;
public class WordReportGenerationService {
private final IuclidFunctionService iuclidFunctionService;
public static String PNG_IMAGE_REGEX = "data:image/png;base64.*";
@Timed("redactmanager_generateWordReport")
@ -94,7 +96,9 @@ public class WordReportGenerationService {
long start = System.currentTimeMillis();
List<ReportRedactionEntry> reportEntries = allReportEntries.stream().filter(entry -> !entry.isSkipped()).collect(Collectors.toList());
List<ReportRedactionEntry> reportEntries = allReportEntries.stream()
.filter(entry -> !entry.isSkipped())
.collect(Collectors.toList());
int sumOfChars = 0;
@ -117,12 +121,12 @@ public class WordReportGenerationService {
}
log.info("Report Generation took: {} for file with id {}, pageCount: {}, entityLogEntryCount: {}, reportName: {}, className: {}",
System.currentTimeMillis() - start,
fileModel.getId(),
fileModel.getNumberOfPages(),
reportEntries.size(),
reportTemplateName,
getClass().getSimpleName());
System.currentTimeMillis() - start,
fileModel.getId(),
fileModel.getNumberOfPages(),
reportEntries.size(),
reportTemplateName,
getClass().getSimpleName());
} catch (Exception e) {
log.error(e.getMessage() + " in file: " + fileModel.getFilename());
@ -139,8 +143,11 @@ public class WordReportGenerationService {
return;
}
for (int j = 0; j < table.getRows().size(); j++) {
for (int i = 0; i < table.getRows().get(j).getTableCells().size(); i++) {
XWPFTableCell cell = table.getRows().get(j).getTableCells().get(i);
for (int i = 0; i < table.getRows()
.get(j).getTableCells().size(); i++) {
XWPFTableCell cell = table.getRows()
.get(j).getTableCells()
.get(i);
if (containsRedactionPlaceholder(cell.getText())) {
table.removeRow(j);
return;
@ -161,6 +168,7 @@ public class WordReportGenerationService {
}
}
}
replaceParagraphForImagePlaceholder(doc.getParagraphs(), imagePlaceholder, img);
}
}
@ -194,11 +202,13 @@ public class WordReportGenerationService {
}
if (placeholderModel.getFileAttributeValueByPlaceholder().containsKey(placeholder)) {
return placeholderModel.getFileAttributeValueByPlaceholder().get(placeholder);
return placeholderModel.getFileAttributeValueByPlaceholder()
.get(placeholder);
}
if (placeholderModel.getDossierAttributesValueByPlaceholder().containsKey(placeholder)) {
return placeholderModel.getDossierAttributesValueByPlaceholder().get(placeholder);
return placeholderModel.getDossierAttributesValueByPlaceholder()
.get(placeholder);
}
return "";
@ -211,15 +221,22 @@ public class WordReportGenerationService {
String paragraphText = p.getText();
if (paragraphText.contains(imagePlaceholder.getPlaceholder())) {
try {
XWPFRun run = p.getRuns()
.get(0);
run.setText("", 0);
run.addBreak();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(img, "jpg", baos);
ImageIO.write(img, imagePlaceholder.getImageType().equals(ImagePlaceholder.ImageType.PNG) ? "png" : "jpg", baos);
baos.flush();
ByteArrayInputStream is = new ByteArrayInputStream(baos.toByteArray());
XWPFRun run = p.getRuns().get(0);
run.setText("", 0);
run.addBreak();
run.addPicture(is, XWPFDocument.PICTURE_TYPE_JPEG, "image.jpg", Units.toEMU(img.getWidth()), Units.toEMU(img.getHeight()));
run.addPicture(is,
imagePlaceholder.getImageType().equals(ImagePlaceholder.ImageType.PNG) ? XWPFDocument.PICTURE_TYPE_PNG : XWPFDocument.PICTURE_TYPE_JPEG,
imagePlaceholder.getImageType().equals(ImagePlaceholder.ImageType.PNG) ? "image.png" : "image.jpg",
Units.toEMU(img.getWidth()),
Units.toEMU(img.getHeight()));
for (int i = p.getRuns().size() - 1; i > 0; i--) {
p.removeRun(i);
@ -285,7 +302,8 @@ public class WordReportGenerationService {
String escapedReplace = Matcher.quoteReplacement(replace);
try {
paragraphText = paragraphText.replaceAll(escapedSearch, escapedReplace);
XWPFRun run = p.getRuns().get(0);
XWPFRun run = p.getRuns()
.get(0);
run.setText(paragraphText, 0);
sumOfChars += paragraphText.length();
} catch (Exception e) {
@ -329,7 +347,8 @@ public class WordReportGenerationService {
private void splitIntoRuns(XWPFParagraph p, String[] stringsOnNewLines, int i) {
p.insertNewRun(i);
XWPFRun newRun = p.getRuns().get(i);
XWPFRun newRun = p.getRuns()
.get(i);
setTextToRun(newRun, stringsOnNewLines[i]);
newRun.addCarriageReturn();
}
@ -352,7 +371,8 @@ public class WordReportGenerationService {
if (i < newLineParagraphs.size()) {
doc.setParagraph(newLineParagraphs.get(i), i);
} else {
paragraphsToRemove.add(doc.getParagraphs().get(i));
paragraphsToRemove.add(doc.getParagraphs()
.get(i));
}
}
@ -376,10 +396,17 @@ public class WordReportGenerationService {
for (XWPFTable tbl : doc.getTables()) {
String tblText = tbl.getText();
if (tblText.contains(PAGE_PLACEHOLDER) || tblText.contains(PARAGRAPH_PLACEHOLDER) || tblText.contains(JUSTIFICATION_PLACEHOLDER) || tblText.contains(EXCERPT_PLACEHOLDER) || tblText.contains(
JUSTIFICATION_PARAGRAPH_PLACEHOLDER) || tblText.contains(JUSTIFICATION_REASON_PLACEHOLDER) || tblText.contains(JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER) || tblText.contains(
JUSTIFICATION_TEXT_PLACEHOLDER) || tblText.contains(SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER) || tblText.contains(
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER)) {
if (tblText.contains(PAGE_PLACEHOLDER)
|| tblText.contains(PARAGRAPH_PLACEHOLDER)
|| tblText.contains(PARAGRAPH_INDEX_PLACEHOLDER)
|| tblText.contains(JUSTIFICATION_PLACEHOLDER)
|| tblText.contains(EXCERPT_PLACEHOLDER)
|| tblText.contains(JUSTIFICATION_PARAGRAPH_PLACEHOLDER)
|| tblText.contains(JUSTIFICATION_REASON_PLACEHOLDER)
|| tblText.contains(JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER)
|| tblText.contains(JUSTIFICATION_TEXT_PLACEHOLDER)
|| tblText.contains(SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER)
|| tblText.contains(SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER)) {
return tbl;
}
}
@ -389,12 +416,22 @@ public class WordReportGenerationService {
private boolean containsRedactionPlaceholder(String text) {
return text.startsWith(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE) || text.startsWith(FILE_ATTRIBUTE_PLACEHOLDER_BASE) || text.contains(FILE_NAME_PLACEHOLDER) || text.contains(
PAGE_PLACEHOLDER) || text.contains(PARAGRAPH_PLACEHOLDER) || text.contains(JUSTIFICATION_PLACEHOLDER) || text.contains(EXCERPT_PLACEHOLDER) || text.contains(
JUSTIFICATION_PARAGRAPH_PLACEHOLDER) || text.contains(JUSTIFICATION_REASON_PLACEHOLDER) || text.contains(REDACTION_VALUE_PLACEHOLDER) || text.contains(
JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER) || text.contains(JUSTIFICATION_TEXT_PLACEHOLDER) || text.contains(
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER) || text.contains(SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER) || text.contains(
REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER);
return text.startsWith(DOSSIER_ATTRIBUTE_PLACEHOLDER_BASE)
|| text.startsWith(FILE_ATTRIBUTE_PLACEHOLDER_BASE)
|| text.contains(FILE_NAME_PLACEHOLDER)
|| text.contains(PAGE_PLACEHOLDER)
|| text.contains(PARAGRAPH_PLACEHOLDER)
|| text.contains(PARAGRAPH_INDEX_PLACEHOLDER)
|| text.contains(JUSTIFICATION_PLACEHOLDER)
|| text.contains(EXCERPT_PLACEHOLDER)
|| text.contains(JUSTIFICATION_PARAGRAPH_PLACEHOLDER)
|| text.contains(JUSTIFICATION_REASON_PLACEHOLDER)
|| text.contains(REDACTION_VALUE_PLACEHOLDER)
|| text.contains(JUSTIFICATION_LEGAL_BASIS_PLACEHOLDER)
|| text.contains(JUSTIFICATION_TEXT_PLACEHOLDER)
|| text.contains(SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER)
|| text.contains(SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER)
|| text.contains(REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER);
}
@ -405,8 +442,11 @@ public class WordReportGenerationService {
if (table != null) {
for (int j = 0; j < table.getRows().size(); j++) {
for (int i = 0; i < table.getRows().get(j).getTableCells().size(); i++) {
XWPFTableCell cell = table.getRows().get(j).getTableCells().get(i);
for (int i = 0; i < table.getRows()
.get(j).getTableCells().size(); i++) {
XWPFTableCell cell = table.getRows()
.get(j).getTableCells()
.get(i);
if (containsRedactionPlaceholder(cell.getText())) {
placeholderCellPos.put(i, getFunctionForPlaceHolder(cell.getText(), foundPlaceHolder, placeholderModel));
} else if (cell.getText().isEmpty()) {
@ -464,6 +504,10 @@ public class WordReportGenerationService {
foundPlaceholders.add(PAGE_PLACEHOLDER);
return input -> String.valueOf(input.getEntry().getPage());
}
if (placeholder.equals(PARAGRAPH_INDEX_PLACEHOLDER)) {
foundPlaceholders.add(PARAGRAPH_INDEX_PLACEHOLDER);
return input -> String.valueOf(input.getEntry().getParagraphIdx());
}
if (placeholder.equals(PARAGRAPH_PLACEHOLDER)) {
foundPlaceholders.add(PARAGRAPH_PLACEHOLDER);
return input -> input.getEntry().getSection();
@ -494,7 +538,8 @@ public class WordReportGenerationService {
}
if (placeholder.equals(REDACTION_VALUE_PLACEHOLDER)) {
foundPlaceholders.add(REDACTION_VALUE_PLACEHOLDER);
return input -> input.getEntry().getValue() != null ? input.getEntry().getValue().replaceAll("\n", " ").replaceAll(" {2}", " ") : input.getEntry().getEntityDisplayName();
return input -> input.getEntry().getValue() != null ? input.getEntry().getValue().replaceAll("\n", " ").replaceAll(" {2}", " ") : input.getEntry()
.getEntityDisplayName();
}
if (placeholder.equals(REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER)) {
foundPlaceholders.add(REDACTION_ENTITY_DISPLAY_NAME_PLACEHOLDER);
@ -503,7 +548,10 @@ public class WordReportGenerationService {
if (placeholder.equals(SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER)) {
foundPlaceholders.add(SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER);
return input -> computePageRanges(input.getRedactionsPerJustificationEntry().getValue().stream().map(ReportRedactionEntry::getPage).collect(Collectors.toSet()));
return input -> computePageRanges(input.getRedactionsPerJustificationEntry().getValue()
.stream()
.map(ReportRedactionEntry::getPage)
.collect(Collectors.toSet()));
}
if (placeholder.equals(SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER)) {
@ -512,12 +560,14 @@ public class WordReportGenerationService {
if (placeholderModel.getFileAttributeValueByPlaceholder().containsKey(placeholder)) {
foundPlaceholders.add(placeholder);
return input -> input.getPlaceholderModel().getFileAttributeValueByPlaceholder().get(placeholder);
return input -> input.getPlaceholderModel().getFileAttributeValueByPlaceholder()
.get(placeholder);
}
if (placeholderModel.getDossierAttributesValueByPlaceholder().containsKey(placeholder)) {
foundPlaceholders.add(placeholder);
return input -> input.getPlaceholderModel().getDossierAttributesValueByPlaceholder().get(placeholder);
return input -> input.getPlaceholderModel().getDossierAttributesValueByPlaceholder()
.get(placeholder);
}
return input -> "";
@ -581,6 +631,10 @@ public class WordReportGenerationService {
private int setText(XWPFTableCell cell, String value) {
// Apache POI automatically adds an initial empty paragraph to a cell resulting in a break line.
if (cell.getParagraphs() != null && !cell.getParagraphs().isEmpty()) {
cell.removeParagraph(0);
}
cell.setVerticalAlignment(XWPFTableCell.XWPFVertAlign.CENTER);
XWPFParagraph addParagraph = cell.addParagraph();
XWPFRun run = addParagraph.createRun();
@ -603,7 +657,9 @@ public class WordReportGenerationService {
private Map<String, List<ReportRedactionEntry>> getRedactionsPerJustification(List<ReportRedactionEntry> reportRedactionEntryList) {
return reportRedactionEntryList.stream().sorted(Comparator.comparing(ReportRedactionEntry::getPage)).collect(Collectors.groupingBy(ReportRedactionEntry::getJustification));
return reportRedactionEntryList.stream()
.sorted(Comparator.comparing(ReportRedactionEntry::getPage))
.collect(Collectors.groupingBy(ReportRedactionEntry::getJustification));
}

View File

@ -19,7 +19,7 @@ import com.iqser.red.service.redaction.report.v1.server.model.ReportRedactionEnt
import com.iqser.red.service.redaction.report.v1.server.model.ReportTemplatesModel;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageServiceAsyncWrapper;
import com.iqser.red.service.redaction.report.v1.server.utils.TemplateCache;
import com.iqser.red.service.redaction.report.v1.server.cache.ReportTemplateCache;
import lombok.AccessLevel;
import lombok.RequiredArgsConstructor;
@ -34,6 +34,7 @@ public class WordReportTemplateService {
ReportStorageService reportStorageService;
ReportStorageServiceAsyncWrapper reportStorageServiceAsyncWrapper;
WordReportGenerationService wordReportGenerationService;
ReportTemplateCache reportTemplateCache;
public CompletableFuture<StoredFileInformation> createWordReportFromTemplateAsync(Dossier dossier,
FileModel fileStatus,
@ -43,7 +44,7 @@ public class WordReportTemplateService {
List<ReportRedactionEntry> reportEntries,
ReportTemplate reportTemplate) {
byte[] wordTemplate = TemplateCache.getTemplate(reportTemplate.getStorageId(), reportStorageService);
byte[] wordTemplate = reportTemplateCache.getTemplate(reportTemplate.getStorageId());
try (ByteArrayInputStream is = new ByteArrayInputStream(wordTemplate)) {
XWPFDocument doc = new XWPFDocument(is);
wordReportGenerationService.generateWordReport(reportEntries, placeholderModel, templateName, doc, fileStatus, dossier, true);
@ -60,7 +61,7 @@ public class WordReportTemplateService {
public void prepareWordReportTemplates(String templateId, ReportTemplate reportTemplate, ReportTemplatesModel reportTemplatesModel) {
byte[] wordTemplate = TemplateCache.getTemplate(reportTemplate.getStorageId(), reportStorageService);
byte[] wordTemplate = reportTemplateCache.getTemplate(reportTemplate.getStorageId());
try (ByteArrayInputStream is = new ByteArrayInputStream(wordTemplate)) {
XWPFDocument doc = new XWPFDocument(is);
MultiFileDocument multiFileDocument = new MultiFileDocument(wordTemplate, doc, templateId, reportTemplate.getFileName(), 0, 0);

View File

@ -8,6 +8,7 @@ import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.apache.commons.io.IOUtils;
@ -15,6 +16,7 @@ import org.springframework.stereotype.Service;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntityLog;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileType;
import com.iqser.red.service.persistence.service.v1.api.shared.mongo.service.EntityLogMongoService;
import com.iqser.red.service.redaction.report.v1.api.model.StoredFileInformation;
import com.iqser.red.storage.commons.service.StorageService;
import com.knecon.fforesight.tenantcommons.TenantContext;
@ -28,6 +30,8 @@ public class ReportStorageService {
private final StorageService storageService;
private final EntityLogMongoService entityLogMongoService;
public String storeObject(String downloadId, byte[] data) {
@ -81,12 +85,14 @@ public class ReportStorageService {
public EntityLog getEntityLog(String dossierId, String fileId, List<String> excludedTypes) {
EntityLog entityLog;
entityLog = storageService.readJSONObject(TenantContext.getTenantId(), StorageIdUtils.getStorageId(dossierId, fileId, FileType.ENTITY_LOG), EntityLog.class);
if (excludedTypes != null) {
entityLog.getEntityLogEntry().removeIf(entry -> excludedTypes.contains(entry.getType()));
Optional<EntityLog> entityLog = entityLogMongoService.findEntityLogByDossierIdAndFileId(dossierId, fileId);
if (entityLog.isEmpty()) {
throw new RuntimeException(String.format("Can't find entity log for dossierId %s and fileId %s", dossierId, fileId));
}
return entityLog;
if (excludedTypes != null) {
entityLog.get().getEntityLogEntry().removeIf(entry -> excludedTypes.contains(entry.getType()));
}
return entityLog.get();
}
}

View File

@ -1,9 +1,8 @@
package com.iqser.red.service.redaction.report.v1.server.storage;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@ -16,10 +15,12 @@ import lombok.RequiredArgsConstructor;
public class ReportStorageServiceAsyncWrapper {
private final ReportStorageService reportStorageService;
private final Executor ioExecutor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
private final TaskExecutor taskExecutor;
public CompletableFuture<String> storeObjectAsync(String downloadId, byte[] data) {
return CompletableFuture.supplyAsync(() -> reportStorageService.storeObject(downloadId, data), ioExecutor);
return CompletableFuture.supplyAsync(() -> reportStorageService.storeObject(downloadId, data), taskExecutor);
}
}

View File

@ -1,31 +0,0 @@
package com.iqser.red.service.redaction.report.v1.server.utils;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import lombok.extern.slf4j.Slf4j;
/**
* This is a caching mechanism for templates to avoid getting a template from storage for each file when we request a download,
* since the templates are provided at startup.
*/
@Slf4j
public class TemplateCache {
private static final Cache<String, byte[]> cache = Caffeine.newBuilder().maximumSize(500).build();
public static byte[] getTemplate(String templateId, ReportStorageService reportStorageService) {
return cache.get(templateId, reportStorageService::getReportTemplate);
}
public static void evictCache(String templateId) {
log.info("Evicting cache for template {}", templateId);
cache.invalidate(templateId);
}
}

View File

@ -7,6 +7,10 @@ fforesight.tenants.remote: true
server:
port: 8080
shutdown: graceful
lifecycle:
base-package: com.iqser.red.service.redaction.report.v1.server
logging.pattern.level: "%5p [${spring.application.name},%X{traceId:-},%X{spanId:-}]"
@ -35,6 +39,23 @@ spring:
max-attempts: 3
max-interval: 15000
prefetch: 1
cache:
type: redis
data:
mongodb:
auto-index-creation: true
# todo: multi-tenancy
database: redaction
host: ${MONGODB_HOST:localhost}
port: ${MONGODB_PORT:27017}
username: ${MONGODB_USER}
password: ${MONGODB_PASSWORD}
redis:
database: 0
host: ${REDIS_HOST:localhost}
port: ${REDIS_PORT:6379}
password: ${REDIS_PASSWORD}
timeout: 60000
management:
endpoint:

View File

@ -0,0 +1,209 @@
package com.iqser.red.service.redaction.report.v1.server;
import static com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration.REPORT_RESPONSE_EXCHANGE;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.amqp.AmqpRejectAndDontRequeueException;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageProperties;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.context.annotation.Import;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.Dossier;
import com.iqser.red.service.redaction.report.v1.api.model.ReportRequestMessage;
import com.iqser.red.service.redaction.report.v1.api.model.ReportResultMessage;
import com.iqser.red.service.redaction.report.v1.server.client.ComponentClient;
import com.iqser.red.service.redaction.report.v1.server.client.DictionaryClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierAttributesClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierClient;
import com.iqser.red.service.redaction.report.v1.server.client.FileAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.client.ReportTemplateClient;
import com.iqser.red.service.redaction.report.v1.server.config.AspectTestConfig;
import com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration;
import com.iqser.red.service.redaction.report.v1.server.queue.ReportMessageReceiver;
import com.iqser.red.service.redaction.report.v1.server.service.EntityLogConverterService;
import com.iqser.red.service.redaction.report.v1.server.service.ExcelReportGenerationService;
import com.iqser.red.service.redaction.report.v1.server.service.GeneratePlaceholderService;
import com.iqser.red.service.redaction.report.v1.server.service.WordReportGenerationService;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
import com.iqser.red.service.redaction.report.v1.server.utils.TestController;
import com.iqser.red.storage.commons.service.StorageService;
import com.knecon.fforesight.lifecyclecommons.LifecycleManager;
import com.knecon.fforesight.tenantcommons.TenantContext;
import com.knecon.fforesight.tenantcommons.TenantsClient;
import lombok.SneakyThrows;
@ExtendWith({SpringExtension.class, MockitoExtension.class})
@SpringBootTest
@Import(AspectTestConfig.class)
public class LifecycleAspectTest {
@Autowired
private ReportMessageReceiver reportMessageReceiver;
@Autowired
private TestController testController;
@Autowired
private LifecycleManager lifecycleManager;
@Mock
private Message mockMessage;
@MockBean
TenantsClient tenantsClient;
@Autowired
private ObjectMapper objectMapper;
@MockBean
private StorageService storageService;
@MockBean
private RabbitTemplate rabbitTemplate;
@MockBean
private ReportStorageService reportStorageService;
@MockBean
private ReportTemplateClient reportTemplateClient;
@MockBean
private FileAttributesConfigClient fileAttributesConfigClient;
@Autowired
private WordReportGenerationService wordReportGenerationService;
@MockBean
private MessagingConfiguration messagingConfiguration;
@Autowired
private EntityLogConverterService entityLogConverterService;
@MockBean
private DossierAttributesConfigClient dossierAttributesConfigClient;
@MockBean
private DossierAttributesClient dossierAttributesClient;
@Autowired
private ExcelReportGenerationService excelTemplateReportGenerationService;
@Autowired
private GeneratePlaceholderService generatePlaceholderService;
@MockBean
private DictionaryClient dictionaryClient;
@MockBean
private DossierClient dossierClient;
@MockBean
private ComponentClient componentClient;
@Mock
private MessageProperties mockMessageProperties;
@BeforeEach
public void setup() {
lifecycleManager.start();
}
@Test
public void testRabbitListenerAspectWhenRunning() throws Exception {
ReportRequestMessage reportRequestMessage = ReportRequestMessage.builder()
.userId("test-user")
.downloadId("123")
.dossierId("dossierId")
.dossierTemplateId("dossierTemplateId")
.build();
String json = objectMapper.writeValueAsString(reportRequestMessage);
prepareDossier();
when(mockMessage.getBody()).thenReturn(json.getBytes());
when(mockMessage.getMessageProperties()).thenReturn(mockMessageProperties);
reportMessageReceiver.receive(mockMessage);
lifecycleManager.stop();
verify(rabbitTemplate, times(1)).convertAndSend(eq(REPORT_RESPONSE_EXCHANGE), eq(TenantContext.getTenantId()), any(ReportResultMessage.class));
}
@Test
public void testRabbitListenerAspectWhenNotRunning() {
lifecycleManager.stop();
assertThrows(AmqpRejectAndDontRequeueException.class, () -> {
reportMessageReceiver.receive(mockMessage);
});
}
@Test
@SneakyThrows
public void testHttpGetAspectWhenRunning() {
String response = testController.testGet();
assertEquals("Test Get", response);
}
@Test
@SneakyThrows
public void testStopLifecycleWhenRunning() {
String response = testController.testGet();
lifecycleManager.stop();
assertEquals("Test Get", response);
}
@Test
public void testHttpGetAspectWhenNotRunning() {
lifecycleManager.stop();
Exception exception = assertThrows(AmqpRejectAndDontRequeueException.class, () -> {
testController.testGet();
});
assertEquals("Application is shutting down, rejecting new messages.", exception.getMessage());
}
@SneakyThrows
private Dossier prepareDossier() {
var testDossier = new Dossier();
testDossier.setName("Test Dossier");
testDossier.setDossierTemplateId("dossierTemplateId");
testDossier.setId("dossierId");
when(dossierClient.getDossierById(eq("dossierId"), anyBoolean(), anyBoolean())).thenReturn(testDossier);
return testDossier;
}
}

View File

@ -10,7 +10,9 @@ import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.amqp.rabbit.core.RabbitAdmin;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistry;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration;
@ -52,6 +54,12 @@ public class PlaceholderTest {
@MockBean
private RabbitTemplate rabbitTemplate;
@MockBean
protected RabbitAdmin rabbitAdmin;
@MockBean
protected RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry;
@MockBean
private ReportTemplateClient reportTemplateClient;
@ -232,11 +240,12 @@ public class PlaceholderTest {
Assertions.assertNotNull(placeholders);
System.out.println(placeholders.size() + " placeholders: " + placeholders);
Assertions.assertFalse(placeholders.isEmpty());
Assertions.assertEquals(12, placeholders.size());
Assertions.assertEquals(13, placeholders.size());
Assertions.assertTrue(placeholders.contains("{{test1}}"));
Assertions.assertTrue(placeholders.contains("{{file.test.1}}"));
Assertions.assertFalse(placeholders.contains("{{test2}}"));
Assertions.assertFalse(placeholders.contains("{{TEST1}}"));
Assertions.assertTrue(placeholders.contains("{{redaction.paragraphIdx}}"));
Assertions.assertFalse(placeholders.contains("{{does.not.exist}}"));
Assertions.assertFalse(placeholders.contains("{{file.test.not.exist}}"));
}
@ -321,11 +330,12 @@ public class PlaceholderTest {
Assertions.assertNotNull(placeholders);
System.out.println(placeholders.size() + " placeholders: " + placeholders);
Assertions.assertFalse(placeholders.isEmpty());
Assertions.assertEquals(6, placeholders.size());
Assertions.assertEquals(7, placeholders.size());
Assertions.assertTrue(placeholders.contains("{{test1}}"));
Assertions.assertTrue(placeholders.contains("{{file.test.1}}"));
Assertions.assertTrue(placeholders.contains("{{file.name}}"));
Assertions.assertTrue(placeholders.contains("{{redaction.paragraph}}"));
Assertions.assertTrue(placeholders.contains("{{redaction.paragraphIdx}}"));
Assertions.assertFalse(placeholders.contains("{{test2}}"));
Assertions.assertFalse(placeholders.contains("{{TEST1}}"));
Assertions.assertFalse(placeholders.contains("{{does.not.exist}}"));

View File

@ -7,6 +7,7 @@ import static com.iqser.red.service.redaction.report.v1.server.service.Placehold
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_DATE_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.FORMAT_TIME_ISO_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.IUCLID_FUNCTION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.REDACTION_ENTITY_HEADLINE_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.service.PlaceholderService.SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER;
import static com.iqser.red.service.redaction.report.v1.server.utils.OsUtils.getTemporaryDirectory;
@ -24,6 +25,7 @@ import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.commons.io.IOUtils;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
@ -31,10 +33,16 @@ import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.xwpf.usermodel.XWPFDocument;
import org.apache.poi.xwpf.usermodel.XWPFParagraph;
import org.apache.poi.xwpf.usermodel.XWPFRun;
import org.apache.poi.xwpf.usermodel.XWPFTable;
import org.apache.poi.xwpf.usermodel.XWPFTableCell;
import org.apache.poi.xwpf.usermodel.XWPFTableRow;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.springframework.amqp.rabbit.core.RabbitAdmin;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistry;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration;
@ -48,6 +56,8 @@ import org.springframework.test.context.junit.jupiter.SpringExtension;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.componentlog.ComponentLog;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.componentlog.ComponentLogEntry;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.componentlog.ComponentLogEntryValue;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntityLog;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntityLogLegalBasis;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.Dossier;
@ -66,6 +76,7 @@ import com.iqser.red.service.redaction.report.v1.server.model.ImagePlaceholder;
import com.iqser.red.service.redaction.report.v1.server.model.PlaceholderModel;
import com.iqser.red.service.redaction.report.v1.server.model.ReportRedactionEntry;
import com.iqser.red.service.redaction.report.v1.server.service.EntityLogConverterService;
import com.iqser.red.service.redaction.report.v1.server.service.ExcelModelFactory;
import com.iqser.red.service.redaction.report.v1.server.service.ExcelReportGenerationService;
import com.iqser.red.service.redaction.report.v1.server.service.GeneratePlaceholderService;
import com.iqser.red.service.redaction.report.v1.server.service.WordReportGenerationService;
@ -98,6 +109,12 @@ public class RedactionReportIntegrationTest {
@MockBean
private RabbitTemplate rabbitTemplate;
@MockBean
protected RabbitAdmin rabbitAdmin;
@MockBean
protected RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry;
@MockBean
private ReportStorageService reportStorageService;
@ -128,6 +145,9 @@ public class RedactionReportIntegrationTest {
@Autowired
private GeneratePlaceholderService generatePlaceholderService;
@Autowired
private ExcelModelFactory excelModelFactory;
@MockBean
private DictionaryClient dictionaryClient;
@ -142,7 +162,7 @@ public class RedactionReportIntegrationTest {
private Dossier prepareDossier() {
var testDossier = new Dossier();
testDossier.setDossierName("Test Dossier");
testDossier.setName("Test Dossier");
testDossier.setDossierTemplateId("dossierTemplateId");
testDossier.setId("dossierId");
@ -158,26 +178,28 @@ public class RedactionReportIntegrationTest {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder().filename("filename").build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, new HashMap<>());
var wordTemplateResource = new ClassPathResource("templates/Justification Appendix A1.docx");
var imageResource = new ClassPathResource("files/exampleImage.jpg");
var wordTemplateResource = new ClassPathResource("templates/AllPlaceholders.docx");
var imageResource = new ClassPathResource("files/signature-40145.png");
var placeholders = buildPlaceHolderModel(Map.of("{{dossier.attribute.ActiveSubstance}}",
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}", IOUtils.toByteArray(imageResource.getInputStream()))));
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}",
IOUtils.toByteArray(imageResource.getInputStream()),
ImagePlaceholder.ImageType.PNG)));
XWPFDocument doc = new XWPFDocument(wordTemplateResource.getInputStream());
wordReportGenerationService.generateWordReport(reportEntries, placeholders, "test", doc, fileModel, dossier, true);
@ -185,6 +207,19 @@ public class RedactionReportIntegrationTest {
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/Justification Appendix A1_justification.docx")) {
fileOutputStream.write(wordReport);
}
for (XWPFTable table : doc.getTables()) {
for (XWPFTableRow row : table.getRows()) {
for (XWPFTableCell cell : row.getTableCells()) {
for (XWPFParagraph paragraph : cell.getParagraphs()) {
String paragraphText = paragraph.getText();
if (!paragraphText.equals("\n")) {
Assertions.assertFalse(paragraphText.startsWith("\n"), "Paragraph starts with an empty line.");
}
}
}
}
}
}
@ -195,7 +230,7 @@ public class RedactionReportIntegrationTest {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder().filename("filename").build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, new HashMap<>());
@ -204,17 +239,19 @@ public class RedactionReportIntegrationTest {
var imageResource = new ClassPathResource("files/exampleImage.jpg");
var placeholders = buildPlaceHolderModel(Map.of("{{dossier.attribute.ActiveSubstance}}",
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}", IOUtils.toByteArray(imageResource.getInputStream()))));
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}",
IOUtils.toByteArray(imageResource.getInputStream()),
ImagePlaceholder.ImageType.JPEG)));
XWPFDocument doc = new XWPFDocument(wordTemplateResource.getInputStream());
wordReportGenerationService.generateWordReport(reportEntries, placeholders, "test", doc, fileModel, dossier, true);
@ -241,7 +278,7 @@ public class RedactionReportIntegrationTest {
assertThat(contentOfParagraphs).isEqualTo(expectedContent);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/IUCLID_Template_justification.docx")) {
try (FileOutputStream fileOutputStream = new FileOutputStream("IUCLID_Template_justification.docx")) {
fileOutputStream.write(wordReportGenerationService.toByteArray(doc));
}
}
@ -274,9 +311,13 @@ public class RedactionReportIntegrationTest {
FileModel fileStatus = FileModel.builder().filename("filename").build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMappingIUCLID.json").getInputStream(), new TypeReference<>() {
});
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, Map.of("signature", "Signature", "logo", "Logo"));
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID,
FILE_ID,
entityLog,
legalBasisMapping,
Map.of("signature", "Signature", "logo", "Logo"));
ClassPathResource templateResource = new ClassPathResource("templates/IUCLID_Template.docx");
XWPFDocument doc = new XWPFDocument(templateResource.getInputStream());
@ -324,13 +365,13 @@ public class RedactionReportIntegrationTest {
Dossier dossier = prepareDossier();
FileModel fileStatus = FileModel.builder().filename("filename").build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, new HashMap<>());
FileModel fileModelSecondFile = FileModel.builder().filename("secondFile").build();
EntityLog entityLogSecondFile = objectMapper.readValue(new ClassPathResource("files/entityLogWithManualRedactions.json").getInputStream(), EntityLog.class);
EntityLog entityLogSecondFile = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<ReportRedactionEntry> reportEntriesSecondFile = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLogSecondFile, legalBasisMapping, new HashMap<>());
ClassPathResource templateResource = new ClassPathResource("templates/Seeds-NewJustificationForm.docx");
@ -355,7 +396,7 @@ public class RedactionReportIntegrationTest {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder().filename("filename").build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, new HashMap<>());
@ -364,17 +405,19 @@ public class RedactionReportIntegrationTest {
var imageResource = new ClassPathResource("files/exampleImage.jpg");
var placeholders = buildPlaceHolderModel(Map.of("{{dossier.attribute.ActiveSubstance}}",
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}", IOUtils.toByteArray(imageResource.getInputStream()))));
"Aktive Substanz \n Test Return",
"{{dossier.attribute.RapporteurMemberState}}",
"Reporter Status",
"{{dossier.attribute.Name}}",
"Dossier Name",
"{{dossier.attribute.Company}}",
"Firma",
"{{dossier.attribute.Date}}",
"2021-11-09T23:00:00.000Z"),
Map.of("{{file.attribute.placeholder}}", "Test"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}",
IOUtils.toByteArray(imageResource.getInputStream()),
ImagePlaceholder.ImageType.JPEG)));
XWPFDocument doc = new XWPFDocument(wordTemplateResource.getInputStream());
wordReportGenerationService.generateWordReport(reportEntries, placeholders, "test", doc, fileModel, dossier, true);
@ -390,9 +433,9 @@ public class RedactionReportIntegrationTest {
public void testExcelTemplateReportGenerationSingleFile() {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder().filename("filename").dossierId(dossier.getDossierId()).fileAttributes(Map.of("TestAttribute", "Lorem Ipsum")).build();
FileModel fileModel = FileModel.builder().filename("filename").dossierId(dossier.getId()).fileAttributes(Map.of("TestAttribute", "Lorem Ipsum")).build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithManualRedactions.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
@ -403,10 +446,10 @@ public class RedactionReportIntegrationTest {
var placeholders = buildPlaceHolderModel(new HashMap<>(), new HashMap<>(), List.of());
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, dossier.getDossierName(), fileModel, excelModel, true);
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, dossier.getName(), fileModel, excelModel, true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/Excel Report_justification.xlsx")) {
@ -421,11 +464,11 @@ public class RedactionReportIntegrationTest {
public void testExcelTemplateReportGenerationMultiFile() {
Dossier dossier = prepareDossier();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID,entityLog, legalBasisMapping, mapOfEntityDisplayName);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, mapOfEntityDisplayName);
FileModel fileModel = FileModel.builder().filename("filename").build();
@ -434,22 +477,26 @@ public class RedactionReportIntegrationTest {
var placeholders = buildPlaceHolderModel(new HashMap<>(), new HashMap<>(), List.of());
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, "dossierName", fileModel, excelModel, false);
EntityLog entityLogSecondFile = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
List<ReportRedactionEntry> reportEntriesSecondFile = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID,entityLogSecondFile, legalBasisMapping, mapOfEntityDisplayName);
List<ReportRedactionEntry> reportEntriesSecondFile = entityLogConverterService.convertAndSort(DOSSIER_ID,
FILE_ID,
entityLogSecondFile,
legalBasisMapping,
mapOfEntityDisplayName);
FileModel fileModelSecondFile = FileModel.builder().filename("secondFile").build();
excelTemplateReportGenerationService.generateExcelReport(reportEntriesSecondFile,
placeholders,
"test",
writeWorkbook,
"dossierName",
fileModelSecondFile,
excelModel,
true);
placeholders,
"test",
writeWorkbook,
"dossierName",
fileModelSecondFile,
excelModel,
true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/Excel Report_multifile.xlsx")) {
@ -460,14 +507,49 @@ public class RedactionReportIntegrationTest {
@Test
@SneakyThrows
public void testExcelPlaceholders() {
void testExcelTemplateQCReportGenerationSingleFile() {
Dossier dossier = prepareDossier();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
FileModel fileModel = FileModel.builder().filename("filename").dossierId(dossier.getId()).fileAttributes(Map.of("TestAttribute", "Lorem Ipsum")).build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithImageHints.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID,entityLog, legalBasisMapping, mapOfEntityDisplayName);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, mapOfEntityDisplayName);
List<ReportRedactionEntry> reportEntriesOnPage16 = reportEntries.stream()
.filter(entry -> entry.getPage() == 16)
.collect(Collectors.toList());
assertThat(reportEntriesOnPage16).hasSize(3);
ClassPathResource templateResource = new ClassPathResource("templates/Excel QC (incl. Skipped Redactions).xlsx");
var placeholders = buildPlaceHolderModel(new HashMap<>(), new HashMap<>(), List.of());
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, dossier.getName(), fileModel, excelModel, true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/Excel Report QC.xlsx")) {
fileOutputStream.write(excelTemplateReport3);
}
System.out.println(getTemporaryDirectory() + "/Excel Report QC.xlsx");
}
@Test
@SneakyThrows
public void testExcelPlaceholders() {
Dossier dossier = prepareDossier();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, mapOfEntityDisplayName);
var imageResource = new ClassPathResource("files/exampleImage.jpg");
FileModel fileModel = FileModel.builder().filename("filename").build();
@ -475,11 +557,13 @@ public class RedactionReportIntegrationTest {
ClassPathResource templateResource = new ClassPathResource("templates/Excel Report_PlaceholderTest.xlsx");
var placeholders = buildPlaceHolderModel(Map.of("{{dossier_attribute.test1}}", "replaced_dossier_test1"),
Map.of("{{file_attribute.test1}}", "replaced_file_test1", "{{file_attribute.test2}}", "replaced_file_test2"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}", IOUtils.toByteArray(imageResource.getInputStream()))));
Map.of("{{file_attribute.test1}}", "replaced_file_test1", "{{file_attribute.test2}}", "replaced_file_test2"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}",
IOUtils.toByteArray(imageResource.getInputStream()),
ImagePlaceholder.ImageType.JPEG)));
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, "dossierName", fileModel, excelModel, true);
@ -491,6 +575,41 @@ public class RedactionReportIntegrationTest {
}
@Test
@SneakyThrows
public void testExcelReportWithPendingEntries() {
Dossier dossier = prepareDossier();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithPendingEntries.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, mapOfEntityDisplayName);
var imageResource = new ClassPathResource("files/exampleImage.jpg");
FileModel fileModel = FileModel.builder().filename("filename").build();
ClassPathResource templateResource = new ClassPathResource("templates/Excel Report_PlaceholderTest.xlsx");
var placeholders = buildPlaceHolderModel(Map.of("{{dossier_attribute.test1}}", "replaced_dossier_test1"),
Map.of("{{file_attribute.test1}}", "replaced_file_test1", "{{file_attribute.test2}}", "replaced_file_test2"),
List.of(new ImagePlaceholder("{{dossier.attribute.Signature}}",
IOUtils.toByteArray(imageResource.getInputStream()),
ImagePlaceholder.ImageType.JPEG)));
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, "dossierName", fileModel, excelModel, true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "Report_Without_Pending_Entries.xlsx")) {
fileOutputStream.write(excelTemplateReport3);
}
}
@Test
@SneakyThrows
public void testScmReport() {
@ -498,12 +617,12 @@ public class RedactionReportIntegrationTest {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder()
.filename("filename")
.dossierId(dossier.getDossierId())
.dossierId(dossier.getId())
.dossierTemplateId(dossier.getDossierTemplateId())
.fileAttributes(Map.of("TestAttribute", "Lorem Ipsum", "1b5ebe26-34f2-4c3b-983d-c0f0a010302c", "402"))
.build();
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/scm/entityLog.json").getInputStream(), EntityLog.class);
EntityLog entityLog = objectMapper.readValue(new ClassPathResource("files/scm/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
List<EntityLogLegalBasis> legalBasisMapping = objectMapper.readValue(new ClassPathResource("files/scm/legalBasisMapping.json").getInputStream(), new TypeReference<>() {
});
ComponentLog componentLog = objectMapper.readValue(new ClassPathResource("files/scm/componentLog.json").getInputStream(), ComponentLog.class);
@ -512,7 +631,7 @@ public class RedactionReportIntegrationTest {
Map<String, String> mapOfEntityDisplayName = createEntityDisplayNames(entityLog);
List<ReportRedactionEntry> reportEntries = entityLogConverterService.convertAndSort(DOSSIER_ID, FILE_ID, entityLog, legalBasisMapping, mapOfEntityDisplayName);
when(componentClient.getComponentLog(dossier.getDossierId(), fileModel.getId(), true)).thenReturn(componentLog);
when(componentClient.getComponentLog(dossier.getId(), fileModel.getId())).thenReturn(componentLog);
when(fileAttributesConfigClient.getFileAttributeConfigs(dossier.getDossierTemplateId())).thenReturn(fileAttributeConfigs);
ClassPathResource templateResource = new ClassPathResource("templates/scm_report.xlsx");
@ -520,10 +639,10 @@ public class RedactionReportIntegrationTest {
var placeholders = buildPlaceHolderModel(new HashMap<>(), Map.of("{{file.attribute.placeholder}}", "OECD Number"), List.of());
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelTemplateReportGenerationService.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, dossier.getDossierName(), fileModel, excelModel, true);
excelTemplateReportGenerationService.generateExcelReport(reportEntries, placeholders, "test", writeWorkbook, dossier.getName(), fileModel, excelModel, true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/scm_report.xlsx")) {
@ -533,6 +652,63 @@ public class RedactionReportIntegrationTest {
}
@Test
@SneakyThrows
public void testComponentLogEntryExceedingMaxValue() {
Dossier dossier = prepareDossier();
FileModel fileModel = FileModel.builder()
.filename("filename")
.dossierId(dossier.getId())
.dossierTemplateId(dossier.getDossierTemplateId())
.fileAttributes(Map.of("TestAttribute", "Lorem Ipsum"))
.build();
ComponentLogEntry componentLogEntry = createLargeComponentLogEntry();
ComponentLog componentLog = new ComponentLog();
componentLog.setComponentLogEntries(List.of(componentLogEntry));
when(componentClient.getComponentLog(dossier.getId(), fileModel.getId())).thenReturn(componentLog);
List<FileAttributeConfig> fileAttributeConfigs = objectMapper.readValue(new ClassPathResource("files/scm/fileAttributes.json").getInputStream(), new TypeReference<>() {
});
when(fileAttributesConfigClient.getFileAttributeConfigs(dossier.getDossierTemplateId())).thenReturn(fileAttributeConfigs);
ClassPathResource templateResource = new ClassPathResource("templates/scm_report.xlsx");
XSSFWorkbook readWorkbook = new XSSFWorkbook(templateResource.getInputStream());
var excelModel = excelModelFactory.calculateExcelModel(readWorkbook.getSheetAt(0), dossier.getDossierTemplateId());
SXSSFWorkbook writeWorkbook = new SXSSFWorkbook();
writeWorkbook.createSheet("Sheet1");
var placeholders = buildPlaceHolderModel(new HashMap<>(), new HashMap<>(), List.of());
excelTemplateReportGenerationService.generateExcelReport(new ArrayList<>(), placeholders, "test", writeWorkbook, dossier.getName(), fileModel, excelModel, true);
byte[] excelTemplateReport3 = excelTemplateReportGenerationService.toByteArray(writeWorkbook);
try (FileOutputStream fileOutputStream = new FileOutputStream(getTemporaryDirectory() + "/scm_report.xlsx")) {
fileOutputStream.write(excelTemplateReport3);
}
System.out.println(getTemporaryDirectory() + "/scm_report.xlsx");
}
private static ComponentLogEntry createLargeComponentLogEntry() {
int length = 35000; // Exceed the max cell length of 32,767
String longValue = "A".repeat(length);
ComponentLogEntryValue componentLogEntryValue = new ComponentLogEntryValue();
componentLogEntryValue.setValue(longValue);
ComponentLogEntry componentLogEntry = new ComponentLogEntry();
componentLogEntry.setName("TestComponent");
componentLogEntry.setValues(List.of(componentLogEntryValue));
return componentLogEntry;
}
private Map<String, String> createEntityDisplayNames(EntityLog entityLog) {
Type t1 = new Type();
@ -545,8 +721,8 @@ public class RedactionReportIntegrationTest {
t3.setLabel("Type 3");
t3.setType("Type 3");
List<Type> ednList = new ArrayList<>(Arrays.asList(t1, t2, t3));
Mockito.when(dictionaryClient.getAllTypesForDossier(Mockito.anyString(), Mockito.anyBoolean())).thenReturn(ednList);
Mockito.when(dictionaryClient.getAllTypesForDossierTemplate(Mockito.any(), Mockito.anyBoolean())).thenReturn(ednList);
Mockito.when(dictionaryClient.getAllTypesForDossier(Mockito.anyString(), Mockito.anyLong(), Mockito.anyBoolean())).thenReturn(ednList);
Mockito.when(dictionaryClient.getAllTypesForDossierTemplate(Mockito.any(), Mockito.anyLong(), Mockito.anyBoolean())).thenReturn(ednList);
Map<String, String> entityDisplayNames = new HashMap<>();
for (var entry : entityLog.getEntityLogEntry()) {
@ -564,14 +740,14 @@ public class RedactionReportIntegrationTest {
private PlaceholderModel buildPlaceHolderModel(Map<String, String> dossierAttributes, Map<String, String> fileAttributes, List<ImagePlaceholder> imagePlaceholders) {
var defaultPlaceHolder = new HashSet<>(Set.of(FILE_NAME_PLACEHOLDER,
FORMAT_DATE_ISO_PLACEHOLDER,
FORMAT_DATE_GER_PLACEHOLDER,
FORMAT_DATE_ENG_PLACEHOLDER,
FORMAT_TIME_ISO_PLACEHOLDER,
DOSSIER_NAME_PLACEHOLDER,
IUCLID_FUNCTION_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER));
FORMAT_DATE_ISO_PLACEHOLDER,
FORMAT_DATE_GER_PLACEHOLDER,
FORMAT_DATE_ENG_PLACEHOLDER,
FORMAT_TIME_ISO_PLACEHOLDER,
DOSSIER_NAME_PLACEHOLDER,
IUCLID_FUNCTION_PLACEHOLDER,
SEEDS_FUNCTION_REDACTION_GROUPED_BY_JUSTIFICATION_PAGES_PLACEHOLDER,
SEEDS_FUNCTION_JUSTIFICATION_PLACEHOLDER, REDACTION_ENTITY_HEADLINE_PLACEHOLDER));
defaultPlaceHolder.addAll(dossierAttributes.keySet());
defaultPlaceHolder.addAll(fileAttributes.keySet());
return new PlaceholderModel(defaultPlaceHolder, imagePlaceholders, dossierAttributes, null, fileAttributes, new HashMap<>());

View File

@ -1,48 +1,8 @@
package com.iqser.red.service.redaction.report.v1.server;
import static com.iqser.red.service.redaction.report.v1.server.utils.OsUtils.getTemporaryDirectory;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
import java.io.FileOutputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration;
import org.springframework.boot.test.autoconfigure.actuate.observability.AutoConfigureObservability;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.FilterType;
import org.springframework.context.annotation.Import;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.ClassPathResource;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntityLog;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntityLogEntry;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntryState;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.EntryType;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.Position;
import com.iqser.red.service.persistence.service.v1.api.shared.model.analysislog.entitylog.*;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.DossierAttributeConfig;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.ReportTemplate;
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.Dossier;
@ -54,14 +14,7 @@ import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemp
import com.iqser.red.service.persistence.service.v1.api.shared.model.dossiertemplate.dossier.file.FileType;
import com.iqser.red.service.redaction.report.v1.api.model.ReportRequestMessage;
import com.iqser.red.service.redaction.report.v1.api.model.StoredFileInformation;
import com.iqser.red.service.redaction.report.v1.server.client.DictionaryClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierAttributesClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.client.DossierClient;
import com.iqser.red.service.redaction.report.v1.server.client.EntityLogClient;
import com.iqser.red.service.redaction.report.v1.server.client.FileAttributesConfigClient;
import com.iqser.red.service.redaction.report.v1.server.client.FileStatusClient;
import com.iqser.red.service.redaction.report.v1.server.client.ReportTemplateClient;
import com.iqser.red.service.redaction.report.v1.server.client.*;
import com.iqser.red.service.redaction.report.v1.server.configuration.MessagingConfiguration;
import com.iqser.red.service.redaction.report.v1.server.service.ReportGenerationService;
import com.iqser.red.service.redaction.report.v1.server.storage.ReportStorageService;
@ -72,10 +25,44 @@ import com.iqser.red.storage.commons.StorageAutoConfiguration;
import com.iqser.red.storage.commons.service.StorageService;
import com.knecon.fforesight.tenantcommons.TenantContext;
import com.knecon.fforesight.tenantcommons.TenantsClient;
import io.micrometer.prometheus.PrometheusMeterRegistry;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.IOUtils;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.amqp.rabbit.core.RabbitAdmin;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistry;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration;
import org.springframework.boot.test.autoconfigure.actuate.observability.AutoConfigureObservability;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.context.annotation.*;
import org.springframework.core.io.ClassPathResource;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import java.io.ByteArrayInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import static com.iqser.red.service.redaction.report.v1.server.utils.OsUtils.getTemporaryDirectory;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.when;
@Slf4j
@ExtendWith(SpringExtension.class)
@ -90,6 +77,12 @@ public class RedactionReportV2IntegrationTest {
@MockBean
private RabbitTemplate rabbitTemplate;
@MockBean
protected RabbitAdmin rabbitAdmin;
@MockBean
protected RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry;
@MockBean
private ReportTemplateClient reportTemplateClient;
@ -206,7 +199,9 @@ public class RedactionReportV2IntegrationTest {
@SneakyThrows
private void processRequest(ReportRequestMessage reportRequestMessage, String fileEnding) {
private List<String> processRequest(ReportRequestMessage reportRequestMessage, String fileEnding) {
List<String> createdOutputFileNames = new ArrayList<>();
var id = reportGenerationService.generateReports(reportRequestMessage);
@ -223,16 +218,18 @@ public class RedactionReportV2IntegrationTest {
try (FileOutputStream fileOutputStream = new FileOutputStream(outputPath)) {
fileOutputStream.write(reportInfo);
}
createdOutputFileNames.add(outputPath);
System.out.println("Created temporary output file: " + outputPath);
i++;
}
return createdOutputFileNames;
}
@SneakyThrows
private void prepareStorage(int id) {
var entityLog = objectMapper.readValue(new ClassPathResource("files/entityLog.json").getInputStream(), EntityLog.class);
var entityLog = objectMapper.readValue(new ClassPathResource("files/entityLogWithParagraphIdx.json").getInputStream(), EntityLog.class);
fileSystemBackedStorageService.storeJSONObject(TenantContext.getTenantId(), StorageIdUtils.getStorageId("dossierId", "fileId" + id, FileType.ENTITY_LOG), entityLog);
when(entityLogClient.getEntityLog("dossierId", "fileId" + id, null, true)).thenReturn(entityLog);
}
@ -315,6 +312,43 @@ public class RedactionReportV2IntegrationTest {
}
@Test
@SneakyThrows
public void testSkippedPlaceholder() {
var reportRequestMessage = prepareFlow(2, "templates/Excel QC (incl. Skipped Redactions).xlsx");
var fileNames = processRequest(reportRequestMessage, ".xlsx");
MetricValidationUtils.validateMetric(prometheusMeterRegistry, "redactmanager_generateReports", 1, null);
MetricValidationUtils.validateMetric(prometheusMeterRegistry, "redactmanager_generateExcelReport", 1, null);
MetricValidationUtils.validateMetric(prometheusMeterRegistry, "redactmanager_calculateExcelModel", 1, null);
MetricValidationUtils.validateMetric(prometheusMeterRegistry, "redactmanager_getReportEntries", 1, null);
AtomicBoolean containsSkipped = new AtomicBoolean(false);
fileNames.forEach(fileName -> {
try (ByteArrayInputStream is = new ByteArrayInputStream(Files.readAllBytes(Paths.get(fileName)))) {
XSSFWorkbook workbook = new XSSFWorkbook(is);
for (Sheet sheet : workbook) {
for (Row row : sheet) {
for (Cell cell : row) {
if ("true".equalsIgnoreCase(cell.getStringCellValue())) {
containsSkipped.set(true);
return;
}
}
}
}
} catch (IOException e) {
System.out.println("Error reading file " + fileName);
}
});
assertTrue(containsSkipped.get());
}
private List<FileModel> createFileModels(int numOfFileModelsToCreate) {
return IntStream.range(1, numOfFileModelsToCreate + 1)

View File

@ -0,0 +1,38 @@
package com.iqser.red.service.redaction.report.v1.server.config;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import com.knecon.fforesight.lifecyclecommons.LifecycleAspect;
import com.knecon.fforesight.lifecyclecommons.LifecycleManager;
import com.knecon.fforesight.lifecyclecommons.LifecycleProperties;
@TestConfiguration
@EnableAspectJAutoProxy
public class AspectTestConfig {
@Bean
public LifecycleProperties lifecycleProperties() {
LifecycleProperties lifecycleProperties = new LifecycleProperties();
lifecycleProperties.setBasePackage("com.knecon");
return lifecycleProperties;
}
@Bean
public LifecycleManager lifecycleManager() {
return new LifecycleManager();
}
@Bean
public LifecycleAspect lifecycleAspect(LifecycleManager lifecycleManager, LifecycleProperties lifecycleProperties) {
return new LifecycleAspect(lifecycleManager, lifecycleProperties);
}
}

View File

@ -12,6 +12,8 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.springframework.amqp.rabbit.core.RabbitAdmin;
import org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistry;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.MockBean;
@ -43,6 +45,12 @@ public class EntityLogConverterServiceTest {
private static final String DOSSIER_ID = "DossierId";
private static final String FILE_ID = "FileId";
@MockBean
protected RabbitAdmin rabbitAdmin;
@MockBean
protected RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry;
@MockBean
TenantsClient tenantsClient;
@ -78,10 +86,10 @@ public class EntityLogConverterServiceTest {
assertNotNull(reportEntries);
assertFalse(reportEntries.isEmpty());
assertEquals(reportEntries.size(), 61);
assertEquals(62, reportEntries.size());
assertEquals(reportEntries.get(0).getPage(), 1);
assertEquals(reportEntries.get(0).getSection(), "[1]: Section: Rule 1/2: Redact CBI");
assertEquals(reportEntries.get(0).getJustification(), "Article 39(e)(3) of Regulation (EC) No 178/2002 ");
assertEquals(reportEntries.get(0).getJustification(), "Article 39(e)(3) of Regulation (EC) No 178/2002 (Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)");
assertEquals(reportEntries.get(0).getJustificationParagraph(), "Article 39(e)(3) of Regulation (EC) No 178/2002");
assertFalse(reportEntries.get(0).isSkipped());
}

View File

@ -19,7 +19,7 @@ public final class OsUtils {
if (isWindows() && StringUtils.isNotBlank(tmpdir)) {
return tmpdir;
}
return "/tmp";
return "/tmp/";
}
}

View File

@ -0,0 +1,17 @@
package com.iqser.red.service.redaction.report.v1.server.utils;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class TestController {
@GetMapping("/test-get")
public String testGet() throws InterruptedException {
Thread.sleep(5000);
return "Test Get";
}
}

View File

@ -6,9 +6,24 @@ management:
endpoints.web.exposure.include: prometheus, health, metrics
metrics.export.prometheus.enabled: true
lifecycle:
base-package: com.iqser.red.service.redaction.report.v1.server
spring:
main:
allow-circular-references: true
cache:
type: SIMPLE
data:
mongodb:
auto-index-creation: true
# todo: multi-tenancy
database: redaction
host: ${MONGODB_HOST:localhost}
port: ${MONGODB_PORT:27017}
username: ${MONGODB_USER}
password: ${MONGODB_PASSWORD}
persistence-service.url: "http://mock.url"
POD_NAME: "redaction-report-service"

View File

@ -1,11 +1,11 @@
filename
Reg (EC) No 1107/2009 Art. 63 (2e)
Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002) commercial links between a producer or importer and the applicant or the authorisation holder, where applicable
relates to: P5 +49 2113 2311 563, P5 +49 2113 2311 560, P5 +81 764770164, P5 +81 6653 44563, P5 Seriknowmobil@co.uk, P5 maximiliamschmitt@arcor.de, P5 maximiliamschmitt@t-online.de, P5 example@mail.com, P5 +27414328992, P5 +274 1432 8990, P5 +274 34223331, P5 +274 1432 8933, P6 +82 122 34188, P6 +82 122 34180, P6 food-industry@korea.com, P6 +275 5678 1234 132, P6 +49 2113 2311 563, P6 +49 2113 2311 560, P6 +81 764770164, P6 +81 6653 44563, P6 Seriknowmobil@co.uk, P6 maximiliamschmitt@arcor.de, P6 maximiliamschmitt@t-online.de, P6 example@mail.com, P6 +27414328992, P6 +274 1432 8990, P6 +274 34223331, P6 +274 1432 8933, P7 +82 122 34188, P7 +82 122 34180, P7 pharma-industry@korea.com
Article 39(e)(3) of Regulation (EC) No 178/2002
Article 39(e)(3) of Regulation (EC) No 178/2002 (Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)
relates to: P1 Logo - non-readable content, P1 Doe J., P2 Michael N., P2 Funnarie B., P2 Feuer A., P3 Desiree, P3 Melanie, P4 library@outlook.com, P4 gordonjcp@msn.com, P4 dinther@comcast.net, P4 kawasaki@me.com, P5 Central Research Industry, P5 Maximiliam Schmitt, P5 +274 1432 8991, P5 493 1223 4592, P5 European Central Institute, P5 Emilia Lockhart, P6 This is a special case, everything between this and the next keyword should be redacted, P6 Central Research Industry, P6 Maximiliam Schmitt, P6 +274 1432 8991, P6 493 1223 4592, P6 European Central Institute, P6 Emilia Lockhart, P7 Dr. Alan Grant, P7 Dr. Alan Grant, P9 Signature - non-readable content, P9 Signature - non-readable content, P9 Müller, P9 Logo - non-readable content

View File

@ -1,289 +0,0 @@
{
"analysisVersion": 1,
"analysisNumber": 1,
"redactionLogEntry": [
{
"id": "28562647b117cbee1737768475ae3607",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 143.0472,
"y": 727.91
},
"width": 10.452833,
"height": 33.710693,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": "Owner (SYN = ",
"textAfter": null,
"comments": null,
"startOffset": 230,
"endOffset": 238,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476409899Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "1f44b16adada98a40be977d224b62d4b",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 546.96716,
"y": 72.89051
},
"width": 10.452818,
"height": 33.71469,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": "the active substance ",
"textAfter": " 6",
"comments": null,
"startOffset": 219,
"endOffset": 227,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476423585Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "91ef6d1848e433516662408700de5790",
"type": "hint_only",
"value": "author",
"reason": null,
"matchedRule": 0,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.98039216,
0.59607846,
0.96862745
],
"positions": [
{
"topLeft": {
"x": 61.0888,
"y": 428.5304
},
"width": 11.591162,
"height": 32.84558,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 147,
"endOffset": 153,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476427502Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": true,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "1c2b8863d0348ec57b337ce2712f0847",
"type": "hint_only",
"value": "author",
"reason": null,
"matchedRule": 0,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.98039216,
0.59607846,
0.96862745
],
"positions": [
{
"topLeft": {
"x": 111.60883,
"y": 72.89054
},
"width": 11.591154,
"height": 32.80046,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 0,
"endOffset": 6,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476432602Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": true,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
}
],
"legalBasis": [
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
],
"dictionaryVersion": 18,
"dossierDictionaryVersion": 1,
"rulesVersion": 1,
"legalBasisVersion": 2
}

View File

@ -1,355 +0,0 @@
{
"analysisVersion": 1,
"analysisNumber": 1,
"redactionLogEntry": [
{
"id": "28562647b117cbee1737768475ae3607",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 143.0472,
"y": 727.91
},
"width": 10.452833,
"height": 33.710693,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": "Owner (SYN = ",
"textAfter": null,
"comments": null,
"startOffset": 230,
"endOffset": 238,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476409899Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "1f44b16adada98a40be977d224b62d4b",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 546.96716,
"y": 72.89051
},
"width": 10.452818,
"height": 33.71469,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": "the active substance ",
"textAfter": " 6",
"comments": null,
"startOffset": 219,
"endOffset": 227,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476423585Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "91ef6d1848e433516662408700de5790",
"type": "hint_only",
"value": "author",
"reason": null,
"matchedRule": 0,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.98039216,
0.59607846,
0.96862745
],
"positions": [
{
"topLeft": {
"x": 61.0888,
"y": 428.5304
},
"width": 11.591162,
"height": 32.84558,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 147,
"endOffset": 153,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476427502Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": true,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "1c2b8863d0348ec57b337ce2712f0847",
"type": "hint_only",
"value": "author",
"reason": null,
"matchedRule": 0,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.98039216,
0.59607846,
0.96862745
],
"positions": [
{
"topLeft": {
"x": 111.60883,
"y": 72.89054
},
"width": 11.591154,
"height": 32.80046,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 0,
"endOffset": 6,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:51.476432602Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": true,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "b1372ac26ef7c9ac846636dc90ddcf05",
"type": "manual",
"value": "Published",
"reason": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"matchedRule": 0,
"rectangle": false,
"legalBasis": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"imported": false,
"redacted": true,
"section": "Text in table",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 157.0195,
"y": 350.69205
},
"width": 13.96788,
"height": -47.821453,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": "",
"textAfter": "",
"comments": null,
"startOffset": 0,
"endOffset": 0,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 2,
"type": "ADDED",
"dateTime": "2023-03-20T11:27:09.936Z"
}
],
"manualChanges": [
{
"annotationStatus": "APPROVED",
"manualRedactionType": "ADD_LOCALLY",
"processedDate": "2023-03-20T11:27:09.944Z",
"requestedDate": "2023-03-20T11:27:09.936Z",
"userId": "98755882-a216-44e6-92c1-0c191ad134ce",
"propertyChanges": {},
"processed": true
}
],
"engines": null,
"reference": null,
"importedRedactionIntersections": [],
"localManualRedaction": true,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": false,
"dossierDictionaryEntry": false
}
],
"legalBasis": [
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
],
"dictionaryVersion": 18,
"dossierDictionaryVersion": 1,
"rulesVersion": 1,
"legalBasisVersion": 2
}

View File

@ -1,639 +0,0 @@
{
"analysisVersion": 1,
"analysisNumber": 1,
"redactionLogEntry": [
{
"id": "ac54224bab3fa173a28c1fc943abcb2b",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 143.0472,
"y": 727.85
},
"width": 10.452833,
"height": 33.710693,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": "Owner (SYN = ",
"textAfter": null,
"comments": null,
"startOffset": 230,
"endOffset": 238,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477812432Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "badbdba336f4d754024bb1f820a61b0a",
"type": "CBI_author",
"value": "Green R.",
"reason": "Author found",
"matchedRule": 10,
"rectangle": false,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"imported": false,
"redacted": true,
"section": "Text in table",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 253.67392,
"y": 162.84514
},
"width": 11.54608,
"height": 36.198288,
"page": 1
}
],
"sectionNumber": 6,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 15,
"endOffset": 23,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477821319Z"
}
],
"manualChanges": [],
"engines": [
"NER",
"DICTIONARY",
"RULE"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "0cbb552705d3294bf579dcfa97ea6766",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 290.2385,
"y": 644.72015
},
"width": 10.981504,
"height": 33.48163,
"page": 1
}
],
"sectionNumber": 9,
"textBefore": "providing access to ",
"textAfter": " specific know-how",
"comments": null,
"startOffset": 393,
"endOffset": 401,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477826438Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "c688622f3544cd4d9c4aafef2c77f15d",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 237.65378,
"y": 302.87067
},
"width": 11.546049,
"height": 37.251373,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": "to Document J ",
"textAfter": " File No",
"comments": null,
"startOffset": 102,
"endOffset": 110,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477828843Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "8ac1b192ab6ca20375149f01a8ce7334",
"type": "CBI_author",
"value": "Akkan Z., Botham J.",
"reason": "Author found",
"matchedRule": 10,
"rectangle": false,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"imported": false,
"redacted": true,
"section": "Text in table",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 214.67392,
"y": 162.85521
},
"width": 11.546062,
"height": 83.2892,
"page": 1
}
],
"sectionNumber": 5,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 15,
"endOffset": 34,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477830937Z"
}
],
"manualChanges": [],
"engines": [
"RULE"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": false,
"dossierDictionaryEntry": false
},
{
"id": "a6a7198a8089e8e06107af5f6a2d1e73",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 290.2385,
"y": 75.3515
},
"width": 10.981504,
"height": 33.536697,
"page": 1
}
],
"sectionNumber": 9,
"textBefore": "2013 ZA1296_10191 *",
"textAfter": " requests data",
"comments": null,
"startOffset": 237,
"endOffset": 245,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477833081Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "2a0f1f825a91671322b6c7adef385367",
"type": "hint_only",
"value": "author",
"reason": null,
"matchedRule": 0,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.98039216,
0.59607846,
0.96862745
],
"positions": [
{
"topLeft": {
"x": 111.60883,
"y": 162.83318
},
"width": 11.591154,
"height": 32.800476,
"page": 1
}
],
"sectionNumber": 2,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 11,
"endOffset": 17,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477835365Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": true,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "a22c85dd1d9005345e48e1e573745567",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 546.96716,
"y": 72.830505
},
"width": 10.452818,
"height": 33.714684,
"page": 1
}
],
"sectionNumber": 9,
"textBefore": "the active substance ",
"textAfter": " 6",
"comments": null,
"startOffset": 196,
"endOffset": 204,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477837359Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "181eec8da7dff594abd8181370a4b2b5",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 290.2385,
"y": 442.6209
},
"width": 10.981504,
"height": 33.48181,
"page": 1
}
],
"sectionNumber": 9,
"textBefore": "information might undermine ",
"textAfter": "s commercial interests",
"comments": null,
"startOffset": 338,
"endOffset": 346,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477839473Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
},
{
"id": "b907ba9ef5993bd3d12d2098cb03759e",
"type": "CBI_address",
"value": "Syngenta",
"reason": "Address found for non vertebrate study",
"matchedRule": 3,
"rectangle": false,
"legalBasis": null,
"imported": false,
"redacted": false,
"section": "Text in table",
"color": [
0.76862746,
0.59607846,
0.98039216
],
"positions": [
{
"topLeft": {
"x": 276.65378,
"y": 302.87067
},
"width": 11.54608,
"height": 37.251373,
"page": 1
}
],
"sectionNumber": 6,
"textBefore": "to Document J ",
"textAfter": " File No",
"comments": null,
"startOffset": 91,
"endOffset": 99,
"imageHasTransparency": false,
"excluded": false,
"sourceId": null,
"changes": [
{
"analysisNumber": 1,
"type": "ADDED",
"dateTime": "2023-02-28T10:05:46.477841627Z"
}
],
"manualChanges": [],
"engines": [
"DICTIONARY"
],
"reference": [],
"importedRedactionIntersections": [],
"localManualRedaction": false,
"manuallyRemoved": false,
"hint": false,
"recommendation": false,
"falsePositive": false,
"image": false,
"dictionaryEntry": true,
"dossierDictionaryEntry": false
}
],
"legalBasis": [
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
],
"dictionaryVersion": 18,
"dossierDictionaryVersion": 1,
"rulesVersion": 1,
"legalBasisVersion": 2
}

View File

@ -1,47 +0,0 @@
[
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(1)",
"description": "any other personal data except for\n a. the name and address of the applicant;\n b. the names of authors of published or publicly available studies supporting such requests; and the names of all participants and observers in meetings of the Scientific Committee and the Scientific Panels, their working groups and any other ad hoc group meeting on the subject matter.",
"reason": "Article 39(e)(1) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
]

View File

@ -1,275 +0,0 @@
{
"analysisVersion": 1,
"redactionLogEntry": [
{
"id": "c65033cf94d9d0011cbb45184225e2c6",
"type": "CBI_address",
"value": "Syngenta Crop Protection AG CH 4002 Basel Switzerland",
"reason": "Address found",
"matchedRule": 3,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "Table in: ",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 301.27,
"y": 687.38
},
"width": 120.99826,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 675.86
},
"width": 69.64087,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 664.34
},
"width": 48.55905,
"height": 10.526819,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 148,
"endOffset": 203,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-20T15:30:11.024443Z"
}
],
"engines": [
"DICTIONARY"
],
"reference": [],
"hint": false,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "dbf366de6f9338b6246a1555058bf652",
"type": "CBI_address",
"value": "Syngenta Crop Protection AG Schwarzwaldalle 215 P.O. Box CH-4002 Basel Switzerland",
"reason": "Address found",
"matchedRule": 3,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "Table in: ",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 301.27,
"y": 745.34
},
"width": 120.99826,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 733.82
},
"width": 86.676025,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 722.3
},
"width": 37.971527,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 710.9
},
"width": 62.99005,
"height": 10.526819,
"page": 1
},
{
"topLeft": {
"x": 301.27,
"y": 699.38
},
"width": 48.55905,
"height": 10.526819,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 17,
"endOffset": 99,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-20T15:30:11.024476Z"
}
],
"engines": [
"DICTIONARY"
],
"reference": [],
"hint": false,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "c0ab0793f4bd54dc02e09606ad58583f",
"type": "false_positive",
"value": "Field",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "Table in: ",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 105.14,
"y": 405.23
},
"width": 24.537277,
"height": 10.929359,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 868,
"endOffset": 873,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-20T15:30:11.024482Z"
}
],
"engines": [
"DICTIONARY"
],
"reference": [],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
}
],
"legalBasis": [
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(1)",
"description": "any other personal data except for\n a. the name and address of the applicant;\n b. the names of authors of published or publicly available studies supporting such requests; and the names of all participants and observers in meetings of the Scientific Committee and the Scientific Panels, their working groups and any other ad hoc group meeting on the subject matter.",
"reason": "Article 39(e)(1) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
],
"dictionaryVersion": 24,
"dossierDictionaryVersion": 1,
"rulesVersion": 1,
"legalBasisVersion": 1
}

View File

@ -1,543 +0,0 @@
{
"analysisVersion": 1,
"redactionLogEntry": [
{
"id": "c1189b63b3293a2cb315c30feaec3a02",
"type": "false_positive",
"value": "Not stated",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 197.93,
"y": 568.67
},
"width": 45.965927,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 1,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 713,
"endOffset": 723,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081665Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "2e9ba259e223dc1205b5c2b6085bec60",
"type": "CBI_author",
"value": "Cheung",
"reason": "Author found",
"matchedRule": 1,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "Materials and methods:",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 153.5,
"y": 403.91
},
"width": 35.3013,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": "ABR89090 (see above, ",
"textAfter": ", 1990 RIP9800436).",
"comments": null,
"startOffset": 425,
"endOffset": 431,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081673Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": false,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "4424ae80a70d7b0ce6d39272ca8b7a19",
"type": "CBI_author",
"value": "Cheung",
"reason": "Author found",
"matchedRule": 1,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "Materials and methods:",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 341.23,
"y": 416.51
},
"width": 35.30124,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": "ABR-89089 (see above, ",
"textAfter": ", 1990, RIP9800437)",
"comments": null,
"startOffset": 374,
"endOffset": 380,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081673Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": false,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "d5b033c5e84d2c9d5a8eb0478d8acf20",
"type": "false_positive",
"value": "September",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "Header",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 459.31998,
"y": 795.04
},
"width": 43.519257,
"height": 10.526819,
"page": 1
}
],
"sectionNumber": 4,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 54,
"endOffset": 63,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081674Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "e80985688619cc8d4fdff3091a7b397a",
"type": "false_positive",
"value": "Syngenta",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 289.84998,
"y": 605.3
},
"width": 42.112946,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 1,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 630,
"endOffset": 638,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081674Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "4745a1125efdc81700e0ab9be95e5c75",
"type": "CBI_author",
"value": "Rollins R.D.",
"reason": "Author found",
"matchedRule": 1,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "",
"color": [
0.5764706,
0.59607846,
0.627451
],
"positions": [
{
"topLeft": {
"x": 409.96426,
"y": 618.02
},
"width": 56.28833,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 1,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": "freezer storage conditions, ",
"textAfter": ", 1995, Study",
"comments": null,
"startOffset": 586,
"endOffset": 598,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081674Z"
}
],
"engines": [
"NER",
"DICTIONARY"
],
"hint": false,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "379492d57d7cf987f7177fb8b31ce0bc",
"type": "recommendation_CBI_author",
"value": "Guideline",
"reason": "Author found",
"matchedRule": 1,
"legalBasis": "Article 39(e)(1) of Regulation (EC) No 178/2002",
"redacted": true,
"section": "",
"color": [
0.5529412,
0.9411765,
0.42352942
],
"positions": [
{
"topLeft": {
"x": 70.944,
"y": 586.82
},
"width": 46.95952,
"height": 10.929359,
"page": 1
}
],
"sectionNumber": 1,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": "No CGA24705/2567, RIP9800438 ",
"textAfter": "(s): Not specified",
"comments": null,
"startOffset": 673,
"endOffset": 682,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081675Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": false,
"recommendation": true,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "39b497ef7a81c0283a01c0156fb523ab",
"type": "false_positive",
"value": "August",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 489.86035,
"y": 740.42
},
"width": 32.82184,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 1,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 90,
"endOffset": 96,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081675Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
},
{
"id": "c0eace8eb34ea01a13a7a55c14dc6e1b",
"type": "false_positive",
"value": "All",
"reason": null,
"matchedRule": 0,
"legalBasis": null,
"redacted": false,
"section": "Materials and methods:",
"color": [
1,
0.80784315,
0.80784315
],
"positions": [
{
"topLeft": {
"x": 70.944,
"y": 378.57
},
"width": 15.0980835,
"height": 11.017679,
"page": 1
}
],
"sectionNumber": 2,
"manual": false,
"status": null,
"manualRedactionType": null,
"manualRedactionUserId": null,
"textBefore": null,
"textAfter": null,
"comments": null,
"startOffset": 451,
"endOffset": 454,
"imageHasTransparency": false,
"excluded": false,
"recategorizationType": null,
"legalBasisChangeValue": null,
"changes": [
{
"type": "ADDED",
"dateTime": "2021-09-09T08:00:53.081675Z"
}
],
"engines": [
"DICTIONARY"
],
"hint": true,
"recommendation": false,
"dictionaryEntry": true,
"image": false,
"dossierDictionaryEntry": false
}
],
"legalBasis": [
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(1)",
"description": "any other personal data except for\n a. the name and address of the applicant;\n b. the names of authors of published or publicly available studies supporting such requests; and the names of all participants and observers in meetings of the Scientific Committee and the Scientific Panels, their working groups and any other ad hoc group meeting on the subject matter.",
"reason": "Article 39(e)(1) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
],
"dictionaryVersion": 28,
"dossierDictionaryVersion": 1,
"rulesVersion": 1,
"legalBasisVersion": 1
}

View File

@ -10,7 +10,7 @@
"value": "Dr. Alan Grant",
"reason": "AUTHOR(S) was found",
"matchedRule": "PII.9.2",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
17,
@ -118,7 +118,7 @@
"value": "Funnarie B.",
"reason": "Author(s) found",
"matchedRule": "CBI.9.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
4,
@ -175,7 +175,7 @@
"value": "Central Research Industry",
"reason": "Found after \"Contact point:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -231,7 +231,7 @@
"value": "food-industry@korea.com",
"reason": "Found after \"E-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -286,7 +286,7 @@
"value": "+274 1432 8991",
"reason": "Found between contact keywords",
"matchedRule": "PII.6.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -342,7 +342,7 @@
"value": "+49 2113 2311 563",
"reason": "Found after \"Phone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -506,7 +506,7 @@
"value": "+274 1432 8933",
"reason": "Found after \"Phone No.\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -562,7 +562,7 @@
"value": "This is a special case, everything between this and the next keyword should be redacted",
"reason": "Found between contact keywords",
"matchedRule": "PII.6.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -682,7 +682,7 @@
"value": "example@mail.com",
"reason": "Found after \"E-mail address:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -801,7 +801,7 @@
"value": "Maximiliam Schmitt",
"reason": "Found between contact keywords",
"matchedRule": "PII.6.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -911,7 +911,7 @@
"value": "gordonjcp@msn.com",
"reason": "Found by Email Regex",
"matchedRule": "PII.1.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
12,
@ -965,7 +965,7 @@
"value": "+81 764770164",
"reason": "Found after \"Tel.:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -1021,7 +1021,7 @@
"value": "Melanie",
"reason": "Author found by \"et al\" regex",
"matchedRule": "CBI.16.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
6,
@ -1076,7 +1076,7 @@
"value": "+274 34223331",
"reason": "Found after \"Telephone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -1132,7 +1132,7 @@
"value": "+49 2113 2311 560",
"reason": "Found after \"Fax:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -1262,7 +1262,7 @@
"value": "+82 122 34188",
"reason": "Found after \"Phone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -1373,7 +1373,7 @@
"value": "maximiliamschmitt@arcor.de",
"reason": "Found after \"Email:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -1429,7 +1429,7 @@
"value": "dinther@comcast.net",
"reason": "Found by Email Regex",
"matchedRule": "PII.1.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
12,
@ -1727,7 +1727,7 @@
"value": "European Central Institute",
"reason": "Found after \"European contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -1857,7 +1857,7 @@
"value": "Müller",
"reason": "Author found",
"matchedRule": "CBI.0.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
22,
@ -1911,7 +1911,7 @@
"value": "maximiliamschmitt@arcor.de",
"reason": "Found after \"Email:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -2021,7 +2021,7 @@
"value": "Melanie",
"reason": "Author found by \"et al\" regex",
"matchedRule": "CBI.16.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
6,
@ -2075,7 +2075,7 @@
"value": "+274 1432 8991",
"reason": "Found between contact keywords",
"matchedRule": "PII.6.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -2513,7 +2513,7 @@
"value": "library@outlook.com",
"reason": "Found by Email Regex",
"matchedRule": "PII.1.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
12,
@ -2675,7 +2675,7 @@
"value": "Desiree",
"reason": "Author found by \"et al\" regex",
"matchedRule": "CBI.16.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
6,
@ -2783,7 +2783,7 @@
"value": "Michael N.",
"reason": "Author(s) found",
"matchedRule": "CBI.9.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
4,
@ -3011,7 +3011,7 @@
"value": "kawasaki@me.com",
"reason": "Found by Email Regex",
"matchedRule": "PII.1.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
12,
@ -3065,7 +3065,7 @@
"value": "Central Research Industry",
"reason": "Found after \"Contact point:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -3121,7 +3121,7 @@
"value": "+81 764770164",
"reason": "Found after \"Tel.:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -3231,7 +3231,7 @@
"value": "+82 122 34180",
"reason": "Found after \"Fax:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
15,
@ -3285,7 +3285,7 @@
"value": "+275 5678 1234 132",
"reason": "Found after \"Tel.:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -3396,7 +3396,7 @@
"value": "Maximiliam Schmitt",
"reason": "Found between contact keywords",
"matchedRule": "PII.6.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -3618,7 +3618,7 @@
"value": "+49 2113 2311 560",
"reason": "Found after \"Fax:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -3784,7 +3784,7 @@
"value": "+274 1432 8933",
"reason": "Found after \"Phone No.\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -3840,7 +3840,7 @@
"value": "+82 122 34188",
"reason": "Found after \"Phone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
15,
@ -3948,7 +3948,7 @@
"value": "+82 122 34180",
"reason": "Found after \"Fax:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -4057,7 +4057,7 @@
"value": "+49 2113 2311 563",
"reason": "Found after \"Phone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -4176,7 +4176,7 @@
"value": "+27414328992",
"reason": "Found after \"Telephone number:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -4232,7 +4232,7 @@
"value": "Doe J.",
"reason": "Author found",
"matchedRule": "CBI.0.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
0,
@ -4397,7 +4397,7 @@
"value": "European Central Institute",
"reason": "Found after \"European contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -4516,7 +4516,7 @@
"value": "example@mail.com",
"reason": "Found after \"E-mail address:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -4572,7 +4572,7 @@
"value": "Emilia Lockhart",
"reason": "Found after \"Alternative contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -4628,7 +4628,7 @@
"value": "+274 1432 8990",
"reason": "Found after \"Fax number:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -4684,7 +4684,7 @@
"value": "pharma-industry@korea.com",
"reason": "Found after \"E-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
15,
@ -4738,7 +4738,7 @@
"value": "+274 1432 8990",
"reason": "Found after \"Fax number:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -4794,7 +4794,7 @@
"value": "+81 6653 44563",
"reason": "Found after \"Tel:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -4906,7 +4906,7 @@
"value": "+81 6653 44563",
"reason": "Found after \"Tel:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -4962,7 +4962,7 @@
"value": "Feuer A.",
"reason": "Author(s) found",
"matchedRule": "CBI.9.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
4,
@ -5072,7 +5072,7 @@
"value": "Seriknowmobil@co.uk",
"reason": "Found after \"E-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -5128,7 +5128,7 @@
"value": "+274 34223331",
"reason": "Found after \"Telephone:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -5184,7 +5184,7 @@
"value": "+27414328992",
"reason": "Found after \"Telephone number:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -5294,7 +5294,7 @@
"value": "Dr. Alan Grant",
"reason": "AUTHOR(S) was found",
"matchedRule": "PII.9.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
16,
@ -5348,7 +5348,7 @@
"value": "maximiliamschmitt@t-online.de",
"reason": "Found after \"e-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
13,
@ -5404,7 +5404,7 @@
"value": "493 1223 4592",
"reason": "Found after \"Contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -5460,7 +5460,7 @@
"value": "maximiliamschmitt@t-online.de",
"reason": "Found after \"e-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -5516,7 +5516,7 @@
"value": "493 1223 4592",
"reason": "Found after \"Contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
13,
@ -5635,7 +5635,7 @@
"value": "Seriknowmobil@co.uk",
"reason": "Found after \"E-mail:\" contact keyword",
"matchedRule": "PII.4.1",
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2e)",
"legalBasis": "links_producer_applicant",
"imported": false,
"containingNodeId": [
14,
@ -5745,7 +5745,7 @@
"value": "Desiree",
"reason": "Author found by \"et al\" regex",
"matchedRule": "CBI.16.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
6,
@ -5799,7 +5799,7 @@
"value": "Emilia Lockhart",
"reason": "Found after \"Alternative contact:\" contact keyword",
"matchedRule": "PII.5.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
14,
@ -5855,7 +5855,7 @@
"value": "Image:Logo",
"reason": "Logo Found",
"matchedRule": "ETC.3.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
1,
@ -5907,7 +5907,7 @@
"value": "Image:Signature",
"reason": "Signature Found",
"matchedRule": "ETC.2.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
21,
@ -5959,7 +5959,7 @@
"value": "Image:Signature",
"reason": "Signature Found",
"matchedRule": "ETC.2.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
21,
@ -6011,7 +6011,7 @@
"value": "Image:Logo",
"reason": "Logo Found",
"matchedRule": "ETC.3.0",
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"imported": false,
"containingNodeId": [
22,
@ -6060,47 +6060,56 @@
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"technicalName": "personal_data_geolocation_article_39e3"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002",
"technicalName": "vertebrate_study_personal_data_geolocation_article_39e2"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "manufacturing_production_process"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "links_producer_applicant"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "commercial_information"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "quantitative_composition"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009",
"technicalName": "specification_impurity"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009",
"technicalName": "results_production_batches"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009",
"technicalName": "composition_plant_protection_product"
}
],
"dictionaryVersion": 21,

View File

@ -1,348 +0,0 @@
{
"redactionLogEntry": [
{
"id": "ba922aabe15e334569dc06b212b3a15b",
"type": "address",
"value": "regina.dorn@syngenta.com",
"reason": "Applicant information was found",
"matchedRule": 6,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1 Identity",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 147.86,
"y": 517.43
},
"width": 122.451065,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 13,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "b841bb2734863a14e95cb15efdf98172",
"type": "address",
"value": "+41 (61) 323 6358",
"reason": "Applicant information was found",
"matchedRule": 6,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1 Identity",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 147.86,
"y": 542.75
},
"width": 83.41362,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 13,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "d39269482f83c64453153826d8de0a5f",
"type": "address",
"value": "Patrick Gardinal",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 404.39
},
"width": 73.40033,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "9ddda875972c0fbcda02ae551cd0d07a",
"type": "address",
"value": "+41 (0) 61 323 60 51",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 328.53
},
"width": 94.33217,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "9e289c70c17cb2b3bf7c96b070d44533",
"type": "address",
"value": "patrick.gardinal@syngenta.com",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 303.21002
},
"width": 141.26324,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "10a4893ec30e268c951b8026e0dc3e11",
"type": "address",
"value": "Syngenta Crop Protection AG",
"reason": "Applicant information was found",
"matchedRule": 6,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1 Identity",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 147.86,
"y": 631.34
},
"width": 133.3113,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 13,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "86b754bc4e70f45038b465a66c0e927b",
"type": "address",
"value": "Regina Dorn",
"reason": "Applicant information was found",
"matchedRule": 6,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1 Identity",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 147.86,
"y": 555.47
},
"width": 57.624176,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 13,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "fd367ab592da09165d1bcd52cfddaf68",
"type": "address",
"value": "Syngenta Crop Protection AG",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 455.03
},
"width": 133.34088,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "3ae8841f1e9a6e022af1880aaa2853ef",
"type": "address",
"value": "Syngenta Crop Protection AG",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 391.77
},
"width": 133.23718,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "027e744e97c8c1cecb974023f8792055",
"type": "address",
"value": "+41 (61) 323 6155",
"reason": "Applicant information was found",
"matchedRule": 6,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1 Identity",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 147.86,
"y": 530.15
},
"width": 83.41362,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 13,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
},
{
"id": "a022f9ecad5b06f9da7bc2ebdcc97c17",
"type": "address",
"value": "+41 (0) 61 323 61 55",
"reason": "Producer was found",
"matchedRule": 7,
"legalBasis": "Reg (EC) No 1107/2009 Art. 63 (2g)",
"redacted": true,
"section": "B.1.1.2 Producer of the plant protection product",
"color": [
0.0,
1.0,
1.0
],
"positions": [
{
"topLeft": {
"x": 148.94,
"y": 315.81
},
"width": 94.33217,
"height": 11.017679,
"page": 4
}
],
"sectionNumber": 14,
"manual": false,
"status": null,
"manualRedactionType": null,
"hint": false
}
],
"dictionaryVersion": 0,
"rulesVersion": 0,
"filename": null
}

View File

@ -1,47 +0,0 @@
[
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(1)",
"description": "any other personal data except for\n a. the name and address of the applicant;\n b. the names of authors of published or publicly available studies supporting such requests; and the names of all participants and observers in meetings of the Scientific Committee and the Scientific Panels, their working groups and any other ad hoc group meeting on the subject matter.",
"reason": "Article 39(e)(1) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
]

View File

@ -1,42 +1,50 @@
[
{
"name": "1. personal data (incl. geolocation)",
"description": "any other personal data except for\n a. the name and address of the applicant;\n b. the names of authors of published or publicly available studies supporting such requests; and the names of all participants and observers in meetings of the Scientific Committee and the Scientific Panels, their working groups and any other ad hoc group meeting on the subject matter.\nand except for personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(1) and Article 39(e)(2) of Regulation (EC) No 178/2002"
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"technicalName": "personal_data_geolocation_article_39e3"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "manufacturing_production_process"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable;",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "links_producer_applicant"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "commercial_information"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "quantitative_composition"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009",
"technicalName": "specification_impurity"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009",
"technicalName": "results_production_batches"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009",
"technicalName": "composition_plant_protection_product"
}
]

View File

@ -2,46 +2,55 @@
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"technicalName": "personal_data_geolocation_article_39e3"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002",
"technicalName": "vertebrate_study_personal_data_geolocation_article_39e2"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "manufacturing_production_process"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "links_producer_applicant"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "commercial_information"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "quantitative_composition"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009",
"technicalName": "specification_impurity"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009",
"technicalName": "results_production_batches"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009",
"technicalName": "composition_plant_protection_product"
}
]
]

View File

@ -1,47 +0,0 @@
[
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
}
]

View File

@ -268,7 +268,7 @@
"value": "www.syngenta.com",
"reason": "PII (Personal Identification Information) found",
"matchedRule": 19,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": true,
"section": "",
"color": [
@ -313,7 +313,7 @@
"value": "N Robinson",
"reason": "Author found",
"matchedRule": 1,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": true,
"section": "",
"color": [
@ -511,7 +511,7 @@
"value": "F J Lewis",
"reason": "PII (Personal Identification Information) found",
"matchedRule": 19,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": true,
"section": "",
"color": [
@ -3157,7 +3157,7 @@
"value": "Thomas",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -3301,7 +3301,7 @@
"value": "Tummon O J",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -4579,7 +4579,7 @@
"value": "Mill",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -4714,7 +4714,7 @@
"value": "Ford, PA",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -4813,7 +4813,7 @@
"value": "Court",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -4858,7 +4858,7 @@
"value": "Mill",
"reason": "Published Information found",
"matchedRule": 18,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": false,
"section": "References",
"color": [
@ -10069,7 +10069,7 @@
"value": "Rectangle",
"reason": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"matchedRule": 0,
"legalBasis": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"legalBasis": "personal_data_geolocation_article_39e3",
"redacted": true,
"section": "Signature",
"color": [
@ -10114,7 +10114,7 @@
"value": "Rectangle",
"reason": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"matchedRule": 0,
"legalBasis": "Article 63(2)(b) of Regulation (EC) No 1107/2009",
"legalBasis": "specification_impurity",
"redacted": true,
"section": "Formula",
"color": [
@ -10158,47 +10158,56 @@
{
"name": "1.1 personal data (incl. geolocation); Article 39(e)(3)",
"description": "(Regulations (EU) 2016/679 and (EU) 2018/1725 shall apply to the processing of personal data carried out pursuant to this Regulation. Any personal data made public pursuant to Article 38 of this Regulation and this Article shall only be used to ensure the transparency of the risk assessment under this Regulation and shall not be further processed in a manner that is incompatible with these purposes, in accordance with point (b) of Article 5(1) of Regulation (EU) 2016/679 and point (b) of Article 4(1) of Regulation (EU) 2018/1725, as the case may be)",
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(3) of Regulation (EC) No 178/2002",
"technicalName": "personal_data_geolocation_article_39e3"
},
{
"name": "1.2 vertebrate study related personal data (incl. geolocation); Article 39(e)(2)",
"description": "personal data (names and addresses) of individuals involved in testing on vertebrate studies or in obtaining toxicological information",
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002"
"reason": "Article 39(e)(2) of Regulation (EC) No 178/2002",
"technicalName": "vertebrate_study_personal_data_geolocation_article_39e2"
},
{
"name": "2. manufacturing or production process",
"description": "the manufacturing or production process, including the method and innovative aspects thereof, as well as other technical and industrial specifications inherent to that process or method, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "manufacturing_production_process"
},
{
"name": "3. links between a producer and applicant",
"description": "commercial links between a producer or importer and the applicant or the authorisation holder, where applicable",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "links_producer_applicant"
},
{
"name": "4. commercial information",
"description": "commercial information revealing sourcing, market shares or business strategy of the applicant",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "commercial_information"
},
{
"name": "5. quantitative composition",
"description": "quantitative composition of the subject matter of the request, except for information which is relevant to the assessment of safety",
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)"
"reason": "Article 63(2)(a) of Regulation (EC) No 1107/2009 (making reference to Article 39 of Regulation EC No 178/2002)",
"technicalName": "quantitative_composition"
},
{
"name": "6. specification of impurity",
"description": "the specification of impurity of the active substance and the related methods of analysis for impurities in the active substance as manufactured, except for the impurities that are considered to be toxicologically, ecotoxicologically or environmentally relevant and the related methods of analysis for such impurities",
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(b) of Regulation (EC) No 1107/2009",
"technicalName": "specification_impurity"
},
{
"name": "7. results of production batches",
"description": "results of production batches of the active substance including impurities",
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(c) of Regulation (EC) No 1107/2009",
"technicalName": "results_production_batches"
},
{
"name": "8. composition of a plant protection product",
"description": "information on the complete composition of a plant protection product",
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009"
"reason": "Article 63(2)(d) of Regulation (EC) No 1107/2009",
"technicalName": "composition_plant_protection_product"
}
],
"dictionaryVersion": 780,

View File

@ -1,4 +1,4 @@
[
[
{
"id": "1b5ebe26-34f2-4c3b-983d-c0f0a010302c",
"csvColumnHeader": "OECD Number",

View File

@ -2,6 +2,7 @@
{
"name": "n-a.",
"description": "n-a.",
"reason": "n-a."
"reason": "n-a.",
"technicalName": "n-a."
}
]

View File

@ -0,0 +1,859 @@
{
"analysisNumber": 2,
"componentRulesVersion": 1,
"componentLogEntries": [
{
"name": "Study_Title",
"componentValues": [
{
"value": "SYN545192 EC (A15457H) - Acute Dermal Toxicity Study in Rats",
"originalValue": "SYN545192 EC (A15457H) - Acute Dermal Toxicity Study in Rats",
"valueDescription": "First found value of type title or else ''",
"componentRuleId": "StudyTitle.0.0",
"componentLogEntityReferences": [
{
"id": "fdc33ec238d48f3f5c9611ea4ba3ae1b",
"type": "title",
"entityRuleId": "DOC.6.1",
"page": 1
}
]
}
]
},
{
"name": "Performing_Laboratory",
"componentValues": [
{
"value": "CiToxLAB Hungary Ltd., Hungary",
"originalValue": "CiToxLAB Hungary Ltd., Hungary",
"valueDescription": "Laboratory name and country found!",
"componentRuleId": "PerformingLaboratory.1.0",
"componentLogEntityReferences": [
{
"id": "f872738237a85170cb073b6220b98e65",
"type": "laboratory_name",
"entityRuleId": "DOC.7.2",
"page": 1
},
{
"id": "1c6dade9e8ea1a6f5e1b83da05f11368",
"type": "laboratory_country",
"entityRuleId": "DOC.7.2",
"page": 1
}
]
}
]
},
{
"name": "Report_Number",
"componentValues": [
{
"value": "12/063-002P",
"originalValue": "12/063-002P",
"valueDescription": "First found value of type report_number or else ''",
"componentRuleId": "ReportNumber.0.0",
"componentLogEntityReferences": [
{
"id": "8ac7d1ad2d1614527bbb055cebb3824f",
"type": "report_number",
"entityRuleId": "DOC.2.0",
"page": 1
}
]
}
]
},
{
"name": "GLP_Study",
"componentValues": [
{
"value": "Yes",
"originalValue": "Yes",
"valueDescription": "Yes if present, No if not",
"componentRuleId": "GLPStudy.0.0",
"componentLogEntityReferences": [
{
"id": "23cacb7ae621b9cdf987f50e3bdd115b",
"type": "glp_study",
"entityRuleId": "DOC.8.0",
"page": 3
},
{
"id": "fcd231292774563c812cddcf4d78c1f5",
"type": "glp_study",
"entityRuleId": "DOC.8.0",
"page": 25
}
]
}
]
},
{
"name": "Test_Guidelines_1",
"componentValues": [
{
"value": "Nº 402: Acute Dermal Toxicity (24/02/1987)",
"originalValue": "Nº 402: Acute Dermal Toxicity (24/02/1987)",
"valueDescription": "OECD Number and guideline year mapped!",
"componentRuleId": "TestGuideline.0.0",
"componentLogEntityReferences": [
{
"id": "911e8b6737c9529a40590dbf6778c24b",
"type": "oecd_guideline_number",
"entityRuleId": "DOC.1.0",
"page": 1
},
{
"id": "962c4942cd62f0103b1bf5759e2e1b1a",
"type": "oecd_guideline_year",
"entityRuleId": "DOC.1.0",
"page": 1
}
]
}
]
},
{
"name": "Test_Guidelines_2",
"componentValues": [
{
"value": "EPA OPPTS 870.1200 (1998), EC 440/2008 (2008)",
"originalValue": "EPA OPPTS 870.1200 (1998), EC 440/2008 (2008)",
"valueDescription": "Joining all values of type ec_guideline, epa_guideline with ', '",
"componentRuleId": "TestGuideline.2.0",
"componentLogEntityReferences": [
{
"id": "506ef85dd4dcb0a2df24d203ad21d220",
"type": "epa_guideline",
"entityRuleId": "DOC.1.0",
"page": 1
},
{
"id": "09c476952cbc59e3661b03cd38d463e7",
"type": "ec_guideline",
"entityRuleId": "DOC.1.0",
"page": 1
}
]
}
]
},
{
"name": "Experimental_Starting_Date",
"componentValues": [
{
"value": "21/03/2012",
"originalValue": "21/03/2012",
"valueDescription": "Convert values of type 'experimental_start_date' to dd/MM/yyyy joined with ', '",
"componentRuleId": "StartDate.0.0",
"componentLogEntityReferences": [
{
"id": "decf41962064ede79b213035f34862b3",
"type": "experimental_start_date",
"entityRuleId": "DOC.3.0",
"page": 7
}
]
}
]
},
{
"name": "Experimental_Completion_Date",
"componentValues": [
{
"value": "04/04/2012",
"originalValue": "04/04/2012",
"valueDescription": "Convert values of type 'experimental_end_date' to dd/MM/yyyy joined with ', '",
"componentRuleId": "CompletionDate.0.0",
"componentLogEntityReferences": [
{
"id": "346a5ce1bd59081c74885078af715284",
"type": "experimental_end_date",
"entityRuleId": "DOC.4.0",
"page": 7
}
]
}
]
},
{
"name": "Certificate_of_Analysis_Batch_Identification",
"componentValues": [
{
"value": "SMU1LP001",
"originalValue": "SMU1LP001",
"valueDescription": "Joining all unique values of type batch_number with ', '",
"componentRuleId": "AnalysisCertificate.0.0",
"componentLogEntityReferences": [
{
"id": "93b7180cb815e41a53a3f1340776dcf2",
"type": "batch_number",
"entityRuleId": "DOC.9.1",
"page": 12
},
{
"id": "e0cb8bec3a8568335eae3d8776237c8f",
"type": "batch_number",
"entityRuleId": "DOC.9.0",
"page": 23
}
]
}
]
},
{
"name": "Species",
"componentValues": [
{
"value": "rats",
"originalValue": "rats",
"valueDescription": "First found value of type species or else ''",
"componentRuleId": "Species.0.0",
"componentLogEntityReferences": [
{
"id": "b897fbedc847ea1d23f349e5350bc8f1",
"type": "species",
"entityRuleId": "DOC.5.2",
"page": 14
},
{
"id": "f5d80ce83f739506f95420a375c8effa",
"type": "species",
"entityRuleId": "DOC.5.2",
"page": 14
},
{
"id": "4f8b024892a7ee7826a7638833ed0de2",
"type": "species",
"entityRuleId": "DOC.5.2",
"page": 14
}
]
}
]
},
{
"name": "Strain",
"componentValues": [
{
"value": "CRL:(WI)",
"originalValue": "CRL:(WI)",
"valueDescription": "First found value of type strain or else ''",
"componentRuleId": "Strain.0.0",
"componentLogEntityReferences": [
{
"id": "b9784d1fa897ade518e807b3e371b275",
"type": "strain",
"entityRuleId": "DOC.5.3",
"page": 14
}
]
}
]
},
{
"name": "Doses_mg_per_kg_bw",
"componentValues": [
{
"value": "A single administration of SYN545192 EC (A15457H) at a dose of 2000 mg/kg body weight was applied dermally to 5 male and 5 female CRL:(WI) rats, followed by a 14-day observation period. The test item was applied as supplied by the Sponsor. The application period was 24 hours. Clinical observations were assessed in all animals at 1 and 5 hours after dosing and daily for 14 days thereafter. Body weight was measured prior to dosing on Day 0 and on Days 7 and 14. All animals were euthanized and subjected to a gross macroscopic examination at the end of the 14-day observation period (Day 14). 14",
"originalValue": "A single administration of SYN545192 EC (A15457H) at a dose of 2000 mg/kg body weight was applied dermally to 5 male and 5 female CRL:(WI) rats, followed by a 14-day observation period. The test item was applied as supplied by the Sponsor. The application period was 24 hours. Clinical observations were assessed in all animals at 1 and 5 hours after dosing and daily for 14 days thereafter. Body weight was measured prior to dosing on Day 0 and on Days 7 and 14. All animals were euthanized and subjected to a gross macroscopic examination at the end of the 14-day observation period (Day 14). 14",
"valueDescription": "Joining all values of type doses_(mg_kg_bw) with ' '",
"componentRuleId": "Necropsy.1.0",
"componentLogEntityReferences": [
{
"id": "c1bf6261c8dc8f8f6aa0d6fe5c6553bd",
"type": "doses_(mg_kg_bw)",
"entityRuleId": "DOC.35.0",
"page": 10
},
{
"id": "ede065ea1725687b77490a484aa2d9e8",
"type": "doses_(mg_kg_bw)",
"entityRuleId": "DOC.35.0",
"page": 10
},
{
"id": "1210a0f7ffd1c0d558d330a09241bc7d",
"type": "doses_(mg_kg_bw)",
"entityRuleId": "DOC.35.0",
"page": 10
}
]
}
]
},
{
"name": "Mortality_Statement",
"componentValues": [
{
"value": "No mortality occurred on the study. 35",
"originalValue": "No mortality occurred on the study. 35",
"valueDescription": "Joining all values of type mortality_statement with ' '",
"componentRuleId": "MortalityStatement.0.0",
"componentLogEntityReferences": [
{
"id": "0ae4b2b53905f238cf17c20df26f209d",
"type": "mortality_statement",
"entityRuleId": "DOC.32.0",
"page": 16
},
{
"id": "849e9e6c36d5c510225587c21d66b18d",
"type": "mortality_statement",
"entityRuleId": "DOC.32.0",
"page": 16
}
]
}
]
},
{
"name": "Weight_Behavior_Changes",
"componentValues": [
{
"value": "No mortality occurred during the study.\nNo adverse clinical signs were observed after treatment with the test item during the 14 day observation period. Erythema was noted in all animals (10/10) on Day 1 only. All animals were symptom free from 2 days after the treatment.\nThere were no treatment related effects on body weight or body weight gain during the observation period.\nThere was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy.\n15",
"originalValue": "No mortality occurred during the study.\nNo adverse clinical signs were observed after treatment with the test item during the 14 day observation period. Erythema was noted in all animals (10/10) on Day 1 only. All animals were symptom free from 2 days after the treatment.\nThere were no treatment related effects on body weight or body weight gain during the observation period.\nThere was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy.\n15",
"valueDescription": "Joining all values of type weight_behavior_changes with '\n'",
"componentRuleId": "WeightBehavior.0.0",
"componentLogEntityReferences": [
{
"id": "952c0822930bf5b8e2026f96cd7d9b37",
"type": "weight_behavior_changes",
"entityRuleId": "DOC.16.0",
"page": 10
},
{
"id": "6ce01344b447329c1afa3f1f846173f0",
"type": "weight_behavior_changes",
"entityRuleId": "DOC.16.0",
"page": 10
},
{
"id": "8ffd807ffd5ca21c759c78f54cd87f4f",
"type": "weight_behavior_changes",
"entityRuleId": "DOC.16.0",
"page": 10
},
{
"id": "9eeb75725d19bc1f13330aa9cdaa3fdc",
"type": "weight_behavior_changes",
"entityRuleId": "DOC.16.0",
"page": 10
},
{
"id": "a74be4f45046f442315f472490e39e59",
"type": "weight_behavior_changes",
"entityRuleId": "DOC.16.0",
"page": 10
}
]
}
]
},
{
"name": "Necropsy_Findings",
"componentValues": [
{
"value": "No treatment related macroscopic findings were observed. There was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy. 39",
"originalValue": "No treatment related macroscopic findings were observed. There was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy. 39",
"valueDescription": "Joining all values of type necropsy_findings with ' '",
"componentRuleId": "Necropsy.0.0",
"componentLogEntityReferences": [
{
"id": "c0c22ea4e6dda938468d4be3102334ae",
"type": "necropsy_findings",
"entityRuleId": "DOC.17.0",
"page": 16
},
{
"id": "799d19b6a371f575c77e988ba7e23063",
"type": "necropsy_findings",
"entityRuleId": "DOC.17.0",
"page": 16
},
{
"id": "24edc84baf21ef6bb37c5b20801d5e84",
"type": "necropsy_findings",
"entityRuleId": "DOC.17.0",
"page": 16
}
]
}
]
},
{
"name": "Deviation_from_the_Guideline",
"componentValues": [
{
"value": "",
"originalValue": "",
"valueDescription": "Joining all values of type with '\n'",
"componentRuleId": "GuidelineDeviation.0.0",
"componentLogEntityReferences": []
}
]
},
{
"name": "Conclusion_LD50_Greater_than",
"componentValues": [
{
"value": "Greater than",
"originalValue": "Greater than",
"valueDescription": "Entity of type 'ld50_greater' found",
"componentRuleId": "Conclusion.1.0",
"componentLogEntityReferences": [
{
"id": "311a441f815daaffc20453a79712a684",
"type": "ld50_greater",
"entityRuleId": "DOC.10.0",
"page": 10
},
{
"id": "107db9f46656e13234c90b9a3a57b694",
"type": "ld50_greater",
"entityRuleId": "DOC.10.0",
"page": 11
},
{
"id": "2c788c7a7118ef9bb18edf7d5e7dbc7b",
"type": "ld50_greater",
"entityRuleId": "DOC.10.0",
"page": 16
}
]
}
]
},
{
"name": "Conclusion_LD50_mg_per_kg",
"componentValues": [
{
"value": "2000",
"originalValue": "2000",
"valueDescription": "Joining all unique values of type ld50_value with ', '",
"componentRuleId": "Conclusion.0.0",
"componentLogEntityReferences": [
{
"id": "fff9617f8d11bd678e1d17ee330d0eef",
"type": "ld50_value",
"entityRuleId": "DOC.10.0",
"page": 10
},
{
"id": "2ef271bd35871a6266f456a7c4360042",
"type": "ld50_value",
"entityRuleId": "DOC.10.0",
"page": 16
}
]
}
]
},
{
"name": "Conclusion_Minimum_Confidence",
"componentValues": [
{
"value": "",
"originalValue": "",
"valueDescription": "Joining all unique values of type with ', '",
"componentRuleId": "Conclusion.2.0",
"componentLogEntityReferences": []
}
]
},
{
"name": "Conclusion_Maximum_Confidence",
"componentValues": [
{
"value": "",
"originalValue": "",
"valueDescription": "Joining all unique values of type with ', '",
"componentRuleId": "Conclusion.3.0",
"componentLogEntityReferences": []
}
]
},
{
"name": "Study_Conclusion",
"componentValues": [
{
"value": "The median lethal dose of SYN545192 EC (A15457H) after a single dermal administration was found to be greater than 2000 mg/kg bw in male and female CRL:(WI) rats.",
"originalValue": "The median lethal dose of SYN545192 EC (A15457H) after a single dermal administration was found to be greater than 2000 mg/kg bw in male and female CRL:(WI) rats.",
"valueDescription": "Joining all values of type study_conclusion with ' '",
"componentRuleId": "StudyConclusion.0.0",
"componentLogEntityReferences": [
{
"id": "2fd7f1df547bbea6932344ded3d0a7bb",
"type": "study_conclusion",
"entityRuleId": "DOC.15.0",
"page": 10
}
]
}
]
},
{
"name": "laboratory_country",
"componentValues": [
{
"value": "Hungary",
"originalValue": "Hungary",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "d469a79eedff7c52b006ae34b36a7ff8",
"type": "laboratory_country",
"entityRuleId": "DOC.7.2",
"page": 1
}
]
},
{
"value": "Hungary",
"originalValue": "Hungary",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "9593a204d3530a8bb90cab1921cf57c1",
"type": "laboratory_country",
"entityRuleId": "DOC.7.2",
"page": 1
}
]
},
{
"value": "United Kingdom",
"originalValue": "United Kingdom",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "1b968e9b5ce3e90e4158c5353facb78f",
"type": "laboratory_country",
"entityRuleId": "DOC.7.1",
"page": 1
}
]
}
]
},
{
"name": "oecd_guideline",
"componentValues": [
{
"value": "OECD 402 (1987)",
"originalValue": "OECD 402 (1987)",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "0adce062717031fcb3130cb4b5121220",
"type": "oecd_guideline",
"entityRuleId": "DOC.1.0",
"page": 1
}
]
}
]
},
{
"name": "study_conclusion",
"componentValues": [
{
"value": "The median lethal dose of SYN545192 EC (A15457H) after a single dermal administration was found to be greater than 2000 mg/kg bw in male and female CRL:(WI) rats.",
"originalValue": "The median lethal dose of SYN545192 EC (A15457H) after a single dermal administration was found to be greater than 2000 mg/kg bw in male and female CRL:(WI) rats.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "f55749c2ff48055d2c2fa7dd08adde62",
"type": "study_conclusion",
"entityRuleId": "DOC.15.0",
"page": 16
}
]
}
]
},
{
"name": "study_design",
"componentValues": [
{
"value": "A single administration of SYN545192 EC (A15457H) at a dose of 2000 mg/kg body weight was applied dermally to 5 male and 5 female CRL:(WI) rats, followed by a 14-day observation period. The test item was applied as supplied by the Sponsor. The application period was 24 hours.",
"originalValue": "A single administration of SYN545192 EC (A15457H) at a dose of 2000 mg/kg body weight was applied dermally to 5 male and 5 female CRL:(WI) rats, followed by a 14-day observation period. The test item was applied as supplied by the Sponsor. The application period was 24 hours.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "5da351be47fa414cfb7f833d3ca87f5e",
"type": "study_design",
"entityRuleId": "DOC.20.0",
"page": 10
}
]
},
{
"value": "Clinical observations were assessed in all animals at 1 and 5 hours after dosing and daily for 14 days thereafter. Body weight was measured prior to dosing on Day 0 and on Days 7 and 14. All animals were euthanized and subjected to a gross macroscopic examination at the end of the 14-day observation period (Day 14).",
"originalValue": "Clinical observations were assessed in all animals at 1 and 5 hours after dosing and daily for 14 days thereafter. Body weight was measured prior to dosing on Day 0 and on Days 7 and 14. All animals were euthanized and subjected to a gross macroscopic examination at the end of the 14-day observation period (Day 14).",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "a7c111ff4a0456cb14456e27df5b155b",
"type": "study_design",
"entityRuleId": "DOC.20.0",
"page": 10
}
]
},
{
"value": "14",
"originalValue": "14",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "aee5552913b8d2b1fb3a57c84d82649b",
"type": "study_design",
"entityRuleId": "DOC.20.0",
"page": 10
}
]
}
]
},
{
"name": "test_results",
"componentValues": [
{
"value": "Study Title: SYN545192 EC (A15457H) - Acute Dermal Toxicity Study in Rats",
"originalValue": "Study Title: SYN545192 EC (A15457H) - Acute Dermal Toxicity Study in Rats",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "15c7b4591b08cb8b9eae17b1e048ed62",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 5
}
]
},
{
"value": "Test Item: SYN545192 EC (A15457H)",
"originalValue": "Test Item: SYN545192 EC (A15457H)",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "bd2f28162667f5307980dcbefceb7b55",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 5
}
]
},
{
"value": "This study has been inspected, and this report audited by the Quality Assurance Unit in compliance with the Principles of Good Laboratory Practice. As far as it can be reasonably established the methods described and the results incorporated in this report accurately reflect the raw data produced during this study.",
"originalValue": "This study has been inspected, and this report audited by the Quality Assurance Unit in compliance with the Principles of Good Laboratory Practice. As far as it can be reasonably established the methods described and the results incorporated in this report accurately reflect the raw data produced during this study.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "8dd6465f6585320429690c8dfec02013",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 5
}
]
},
{
"value": "All inspections, data reviews and the report audit were reported in written form to the study director and to management. The dates of such inspections and of the report audit are given below:",
"originalValue": "All inspections, data reviews and the report audit were reported in written form to the study director and to management. The dates of such inspections and of the report audit are given below:",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "142b22eb9e0c50104a33a939a1a7b3cf",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 5
}
]
},
{
"value": "4.2",
"originalValue": "4.2",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "4280bb81c5a58bedf7f2578ffbac9482",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 5
}
]
},
{
"value": "No mortality occurred on the study.",
"originalValue": "No mortality occurred on the study.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "ac4cf632670c1e6b9f29a36a2f3a1545",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "35",
"originalValue": "35",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "7ef3c9a644f708038fd627ce289b6fec",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "There were no adverse clinical signs noted in any animals throughout the study.",
"originalValue": "There were no adverse clinical signs noted in any animals throughout the study.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "c6988066e4ea1b9fca609f2d04bc68ee",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "36",
"originalValue": "36",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "41e6e61eb69d675d15ad0510a58e5ac8",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "Erythema was noted in all animals (10/10) on Day 1. All animals were symptom free from 2 days after the treatment.",
"originalValue": "Erythema was noted in all animals (10/10) on Day 1. All animals were symptom free from 2 days after the treatment.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "29fa49dda60b8e8a243e2f46a9a9b66a",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "37",
"originalValue": "37",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "a49264ce0b7d01cb9e5ac6c439fc21b9",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "There were no effects on either body weight or body weight gain in any animal.",
"originalValue": "There were no effects on either body weight or body weight gain in any animal.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "89500fb07a64791681bc4e93edf5f950",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "38",
"originalValue": "38",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "f0cd5807ab584affe57e44d8744256ef",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "No treatment related macroscopic findings were observed.",
"originalValue": "No treatment related macroscopic findings were observed.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "aee6af989b1c6da3cc371929fe92b707",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "There was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy.",
"originalValue": "There was no evidence of the test item-related observations at a dose level of 2000 mg/kg bw at necropsy.",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "971796cd69839b6715ef11dadff12960",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
},
{
"value": "39",
"originalValue": "39",
"valueDescription": "Unmapped Entity",
"componentRuleId": "DefaultComponents.999.0",
"componentLogEntityReferences": [
{
"id": "a3957a14b4e31bcfe42ba11e85cce322",
"type": "test_results",
"entityRuleId": "DOC.24.1",
"page": 16
}
]
}
]
}
]
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 626 KiB

View File

@ -0,0 +1,17 @@
<configuration>
<springProperty scope="configuration" name="logType" source="logging.type"/>
<springProperty scope="context" name="application.name" source="spring.application.name"/>
<springProperty scope="context" name="version" source="project.version"/>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<root level="INFO">
<appender-ref ref="${logType}"/>
</root>
</configuration>