diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/en_doc.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/en_doc.md new file mode 100644 index 000000000..cad20e2c7 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/en_doc.md @@ -0,0 +1,44 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +```toml +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Mermaid diagram + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_gt.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_gt.md new file mode 100644 index 000000000..f46070156 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_gt.md @@ -0,0 +1,45 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +```toml +# Extra line +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_lt.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_lt.md new file mode 100644 index 000000000..e08baa70b --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_lt.md @@ -0,0 +1,45 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +The following block is missing first line: + +```toml +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_not_translated.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_not_translated.md new file mode 100644 index 000000000..cacb9546d --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_not_translated.md @@ -0,0 +1,44 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +```toml +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_translated.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_translated.md new file mode 100644 index 000000000..d03dca53e --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_translated.md @@ -0,0 +1,44 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +```toml +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|требует| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_gt.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_gt.md new file mode 100644 index 000000000..e77050cc9 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_gt.md @@ -0,0 +1,50 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +```toml +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +Extra code block + +``` +$ cd my_project +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_lt.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_lt.md new file mode 100644 index 000000000..918cb883f --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_lt.md @@ -0,0 +1,41 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +Missing code block... + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code.md new file mode 100644 index 000000000..88aed900d --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code.md @@ -0,0 +1,46 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +The following block has wrong language code (should be TOML): + +```yaml +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code_2.md b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code_2.md new file mode 100644 index 000000000..a7fbb39f5 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code_2.md @@ -0,0 +1,46 @@ +# Code blocks { #code-blocks } + +Some text + +```python +# This is a sample Python code block +def hello_world(): + # Comment with indentation + print("Hello, world!") # Print greeting +``` + +Some more text + +The following block has wrong language code (should be TOML): + +``` +# This is a sample TOML code block +title = "TOML Example" # Title of the document +``` + +And more text + +```console +// Use the command "live" and pass the language code as a CLI argument +$ python ./scripts/docs.py live es + +[INFO] Serving on http://127.0.0.1:8008 +[INFO] Start watching changes +[INFO] Start detecting changes +``` + +And even more text + +```json +{ + // This is a sample JSON code block + "greeting": "Hello, world!" // Greeting +} +``` + +Диаграма Mermaid + +```mermaid +flowchart LR + stone(philosophers-stone) -->|requires| harry-1[harry v1] +``` diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_lines_number_mismatch.py b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_lines_number_mismatch.py new file mode 100644 index 000000000..3faa2a7f3 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_lines_number_mismatch.py @@ -0,0 +1,54 @@ +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from scripts.translation_fixer import cli + +data_path = Path( + "scripts/tests/test_translation_fixer/test_code_blocks/data" +).absolute() + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_lines_number_gt.md")], + indirect=True, +) +def test_gt(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_lines_number_gt.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Code block has different number of lines than the original block (5 vs 4)" + ) in result.output + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_lines_number_lt.md")], + indirect=True, +) +def test_lt(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + # assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_lines_number_lt.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Code block has different number of lines than the original block (3 vs 4)" + ) in result.output diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_mermaid.py b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_mermaid.py new file mode 100644 index 000000000..3300671d9 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_mermaid.py @@ -0,0 +1,53 @@ +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from scripts.translation_fixer import cli + +data_path = Path( + "scripts/tests/test_translation_fixer/test_code_blocks/data" +).absolute() + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_mermaid_translated.md")], + indirect=True, +) +def test_translated(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 0, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_mermaid_translated.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert ( + "Skipping mermaid code block replacement. This should be checked manually." + ) in result.output + + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_mermaid_not_translated.md")], + indirect=True, +) +def test_not_translated(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 0, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_mermaid_not_translated.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert ( + "Skipping mermaid code block replacement. This should be checked manually." + ) not in result.output diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_number_mismatch.py b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_number_mismatch.py new file mode 100644 index 000000000..b05dac900 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_number_mismatch.py @@ -0,0 +1,56 @@ +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from scripts.translation_fixer import cli + +data_path = Path( + "scripts/tests/test_translation_fixer/test_code_blocks/data" +).absolute() + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_number_gt.md")], + indirect=True, +) +def test_gt(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_number_gt.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Number of code blocks does not match the number " + "in the original document (6 vs 5)" + ) in result.output + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_number_lt.md")], + indirect=True, +) +def test_lt(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + # assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_number_lt.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Number of code blocks does not match the number " + "in the original document (4 vs 5)" + ) in result.output diff --git a/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_wrong_lang_code.py b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_wrong_lang_code.py new file mode 100644 index 000000000..5546d8e61 --- /dev/null +++ b/scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_wrong_lang_code.py @@ -0,0 +1,55 @@ +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from scripts.translation_fixer import cli + +data_path = Path( + "scripts/tests/test_translation_fixer/test_code_blocks/data" +).absolute() + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_wrong_lang_code.md")], + indirect=True, +) +def test_wrong_lang_code_1(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_wrong_lang_code.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Code block has different language than the original block ('yaml' vs 'toml')" + ) in result.output + + + +@pytest.mark.parametrize( + "copy_test_files", + [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_wrong_lang_code_2.md")], + indirect=True, +) +def test_wrong_lang_code_2(runner: CliRunner, root_dir: Path, copy_test_files): + result = runner.invoke( + cli, + ["fix-pages", "docs/lang/docs/doc.md"], + ) + assert result.exit_code == 1, result.output + + fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text() + expected_content = Path(f"{data_path}/translated_doc_wrong_lang_code_2.md").read_text() + + assert fixed_content == expected_content # Translated doc remains unchanged + assert "Error processing docs/lang/docs/doc.md" in result.output + assert ( + "Code block has different language than the original block ('' vs 'toml')" + ) in result.output diff --git a/scripts/tests/test_translation_fixer/test_complex_doc/data/en_doc.md b/scripts/tests/test_translation_fixer/test_complex_doc/data/en_doc.md index 3582b0491..e64b149da 100644 --- a/scripts/tests/test_translation_fixer/test_complex_doc/data/en_doc.md +++ b/scripts/tests/test_translation_fixer/test_complex_doc/data/en_doc.md @@ -103,6 +103,39 @@ def hello_world():// Print greeting print("Hello, world!") //Print greeting without space after slashes ``` +## Code blocks with quadruple backticks { #code-blocks-with-quadruple-backticks } + +The following code block uses quadruple backticks. + +````python +# Hello world function +def hello_world(): + print("Hello, world!") # Print greeting +```` + +### Backticks number mismatch is fixable { #backticks-number-mismatch-is-fixable } + +The following code block has triple backticks in the original document, but quadruple backticks in the translated document. +It will be fixed by the fixer tool (will convert to triple backticks). + +```Python +# Some Python code +``` + +### Triple backticks inside quadruple backticks { #triple-backticks-inside-quadruple-backticks } + +Comments inside nested code block will NOT be preserved. + +```` +Here is a code block with quadruple backticks that contains triple backticks inside: + +```python +# This is a sample Python code block +def hello_world(): + print("Hello, world!") # Print greeting +``` + +```` # Code includes { #code-includes } diff --git a/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc.md b/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc.md index d92480ae8..5a1fdb40a 100644 --- a/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc.md +++ b/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc.md @@ -101,6 +101,40 @@ def hello_world():// Печать приветствия print("Hello, world!") //Печать приветствия без пробела после слешей ``` +## Блок кода с четырёхкратными обратными кавычками { #code-blocks-with-quadruple-backticks } + +Следующий блок кода содержит четырёхкратные обратные кавычки. + +````python +# Функция приветствия +def hello_world(): + print("Hello, world") # Печать приветствия +```` + +### Несоответствие обратных кавычек фиксится { #backticks-number-mismatch-is-fixable } + +Следующий блок кода имеет тройные обратные кавычки в оригинальном документе, но четырёхкратные обратные кавычки в переведённом документе. +Это будет исправлено инструментом исправления (будет преобразовано в тройные обратные кавычки). + +````Python +# Немного кода на Python +```` + +### Блок кода в тройных обратных кавычка внутри блока кода в четырёхкратных обратных кавычках { #triple-backticks-inside-quadruple-backticks } + +Комментарии внутри вложенного блока кода в тройных обратных кавычках НЕ БУДУТ сохранены. + +```` +Here is a code block with quadruple backticks that contains triple backticks inside: + +```python +# Этот комментарий НЕ будет сохранён +def hello_world(): + print("Hello, world") # Как и этот комментарий +``` + +```` + # Включения кода { #code-includes } ## Простые включения кода { #simple-code-includes } diff --git a/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc_expected.md b/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc_expected.md index b27e48632..5fddcdfee 100644 --- a/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc_expected.md +++ b/scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc_expected.md @@ -101,6 +101,40 @@ def hello_world():// Print greeting print("Hello, world!") //Print greeting without space after slashes ``` +## Блок кода с четырёхкратными обратными кавычками { #code-blocks-with-quadruple-backticks } + +Следующий блок кода содержит четырёхкратные обратные кавычки. + +````python +# Функция приветствия +def hello_world(): + print("Hello, world!") # Печать приветствия +```` + +### Несоответствие обратных кавычек фиксится { #backticks-number-mismatch-is-fixable } + +Следующий блок кода имеет тройные обратные кавычки в оригинальном документе, но четырёхкратные обратные кавычки в переведённом документе. +Это будет исправлено инструментом исправления (будет преобразовано в тройные обратные кавычки). + +```Python +# Немного кода на Python +``` + +### Блок кода в тройных обратных кавычка внутри блока кода в четырёхкратных обратных кавычках { #triple-backticks-inside-quadruple-backticks } + +Комментарии внутри вложенного блока кода в тройных обратных кавычках НЕ БУДУТ сохранены. + +```` +Here is a code block with quadruple backticks that contains triple backticks inside: + +```python +# This is a sample Python code block +def hello_world(): + print("Hello, world!") # Print greeting +``` + +```` + # Включения кода { #code-includes } ## Простые включения кода { #simple-code-includes }