Marmelator/redaction-service
1
0
Fork 0
Code Issues Pull Requests 14 Actions Packages Projects Releases 622 Wiki Activity
1,183 Commits 109 Branches 2,256 Tags
Commit Graph

77 Commits

Author SHA1 Message Date
Kilian Schuettler
86aa9ab3f7 RED-6009 - Document Tree Structure 
* implemented table merging when no headers present
 2023-05-04 16:46:07 +02:00
Kilian Schuettler
a9e3c1e467 RED-6009 - Document Tree Structure 
* removed ner entity validation due to poor performance
* filtered ner entities during reanalysis to improve performance
 2023-05-04 11:31:40 +02:00
Kilian Schuettler
a78a18fe6d RED-6009 - Document Tree Structure 
* moved NER Entity combination to drools
 2023-05-03 16:01:46 +02:00
Kilian Schuettler
fe818b65d0 RED-6009 - Document Tree Structure 
* fix rebased tests
 2023-05-02 19:20:52 +02:00
Kilian Schuettler
7f0699e3bf RED-6009 - Document Tree Structure 
* minor rule fix
 2023-05-02 17:46:48 +02:00
Kilian Schuettler
fe29e0ab3a RED-6009: Document Tree Structure 
* added a test for removing entities by ManualResizeRedaction in ReAnalysis
* Fixed Set logic when removing and readding a resized entity
* Refactored RedactionLog change computation
 2023-05-02 17:46:48 +02:00
Kilian Schuettler
8682b01861 RED-6009: Document Tree Structure 
* added a TableFactory to refactor DocumentGraphFactory
* updated all enabled tests to run with new structure
* moved PdfTableCells to classification package
 2023-05-02 17:46:46 +02:00
Kilian Schuettler
1f7caeebf6 RED-6009: Document Tree Structure 
*renamed old TextBlock and Table Classes
*added subsections to enable separate rules for tables and text-only sections
*added ManualRedaction Tests
*finished default rules
 2023-05-02 17:45:45 +02:00
Kilian Schuettler
86fe535460 RED-6009: Document Tree Structure 
* refactored package structure
 2023-05-02 17:45:44 +02:00
Kilian Schuettler
bda1cc4372 RED-6369: Rules Refactor 
* minor renaming
* added NerEntityAdapter to include the Ner Entities
* moved combine logic outside of rules
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
5603f3265e RED-6369: Rules Refactor 
* added id to ImageNode
* added Manual Redaction Handling to Rules
* minor rename
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
5628db7a36 RED-6369: Rules Refactor 
*removed manualredaction handling for further analysis of how it actually works
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
86b5059d77 RED-6369: Rules Refactor 
* marked entities as resized if a resizeRedaction with same id exists
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
2278195318 RED-6369: Rules Refactor 
* refactored documentGraph.tableOfContents to include document as root
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
dcc2319a00 RED-6369: Rules Refactor 
* refactored DocumentData to 4 different jsons to make reading only text possible
* wip integration of new rules into workflow
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
1fd483a6d4 RED-6369: Rules Refactor 
*added some new Rules and Methods to support them
*integrated local dictionary searches into ruleflow using agenda-groups
*Generalized methods to be able to execute on any SemanticNode
*made Documentgraph implement SemanticNode
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
d36ea08573 RED-6368: Include Tables and Images into Prototype Document Structure 
*added some starting rules
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
6d31868f5d RED-6093: Prototype document structure 
*refactored File Structure
*started refactor of original rules
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
632a1646e3 RED-6093: Prototype find entities in rules 
*added improved string to text position mapping
 2023-05-02 17:44:47 +02:00
Kilian Schuettler
56d7fa950c RED-6093: Prototype find entities in rules 
*added a prototype paragraph rule
 2023-05-02 17:44:47 +02:00
deiflaender
525b8d6434 RED-6093: Prototype find entities in rules 2023-05-02 17:44:44 +02:00
deiflaender
16b04b5918 RED-5276: Fixed strange behavior of text parsing for tables example document 2023-01-31 11:03:55 +01:00
deiflaender
503df78f88 RSS-145: Added rules to add new FileAttributes 2022-10-28 12:33:45 +02:00
deiflaender
58c3d15b78 RSS-92: Fixed excludeHeadlines on tables 2022-09-12 12:58:15 +02:00
deiflaender
a5f27cfa4c RSS-86: Added new rule function redactLineAfterAcrossColumns 2022-09-08 13:05:13 +02:00
deiflaender
c29c5eef0d RSS-42: Made redactBetween rule more generic 2022-09-07 14:31:08 +02:00
deiflaender
6bc5a6a135 RED-3974: Do not add header to header row 2022-08-12 11:29:12 +02:00
deiflaender
f84a366328 RED-3974: Use first row as header if header detection does not find a header 2022-08-11 11:54:25 +02:00
Kresnadi Budisantoso
5b5d898b93 RED-3761 Fix expandByRegEx and expandByPrefixRegEx 
Adjusted the overlap test to ignore if the compared entities are subsets
of each other (start or end position is equal). The `removeEntitiesContainedInLarger`
instruction will remove the subset entity.
 2022-04-11 11:54:30 +02:00
deiflaender
88e6ac3d22 RED-2836: Enabled false positives per dictionary 2022-03-21 12:47:27 +01:00
deiflaender
f32d9a5267 Improved rule to combine AI matches 2022-01-21 12:53:33 +01:00
deiflaender
6e720a15c2 RED-3133: Add and combine AI Entities in rules 2022-01-20 12:11:45 +01:00
aoezyetimoglu
1b9251b0a2 RED-3061: Logic for value matcher for initials expansion rule is inverted 2021-12-14 11:10:59 +01:00
Philipp Schramm
cda8e51d9d Pull request #286: Bugfix/RED-2845 3.0 
Merge in RED/redaction-service from bugfix/RED-2845_3.0 to master

* commit '1c982a40422a16e9caa8d264e4221524c7e1c92b':
  RED-2845 Bugfix: Avoid the expansion if it would result in a redaction overlap and bugfix in RegExp in drools
 2021-12-02 14:07:45 +01:00
Philipp Schramm
1c982a4042 RED-2845 Bugfix: Avoid the expansion if it would result in a redaction overlap and bugfix in RegExp in drools 2021-12-02 14:02:31 +01:00
deiflaender
f37f3b2dcf RED-2862: Added possibility to ignore dictionaries by rule 2021-12-01 12:40:59 +01:00
aoezyetimoglu
d1317c5bd4 RED-2841: INC6207970 Rule for initials expansion should be applied only to dictionary entries without whitespaces 2021-11-29 12:38:53 +01:00
Dominique Eifländer
6f930b1813 RED-2125: Enabled possibility to reference annotations in rules 2021-09-13 12:43:26 +02:00
Dominique Eifländer
6ae6d467fc RED-1445: Enabled to use file attributes in rules 2021-06-29 10:24:51 +02:00
Dominique Eifländer
48beb984ca RED-1334: Enabled to use dictionaries per dossier 2021-04-28 15:11:11 +02:00
Dominique Eifländer
55ba351362 RED-1260: Enabled to add rules and manuel redaction actions for images 2021-04-15 12:51:29 +02:00
Dominique Eifländer
35f3582d08 Expand CBI Authors with firstname initials 2021-02-02 12:20:40 +01:00
Dominique Eifländer
7dfaf604e7 Fixed recognize rules and exceptions for redactAndRecommend, fixed wrong order of rules(published information) 2021-01-25 11:20:35 +01:00
Dominique Eifländer
33795527fd RED-934: Added rule to redact purity and et al Authors will be redacted where they are found 2021-01-20 13:18:37 +01:00
Dominique Eifländer
f3b0eb2911 Recommend CBI_address in Test Organism/Animals sections 2021-01-12 11:30:40 +01:00
Dominique Eifländer
633fb403e0 Fixed RegEx for et al. recommendations 2021-01-07 13:07:21 +01:00
Dominique Eifländer
de725a630c RED-727: Added possibility to redact/addRecommendations by regEx in rules. Added email regEx and et al. author recommendation regEx 2021-01-06 14:53:49 +01:00
deiflaender
608ea4bbcc RED-824: Add author recommendations based on vertebrate study tables 2020-12-07 15:53:20 +01:00
deiflaender
4ef6e0e2ef RED-740: Improved section recognition 2020-11-27 15:39:31 +01:00
deiflaender
d466d9b032 Rename sponsor dictionary to CBI_sponsor 2020-11-26 12:09:19 +01:00