Add tests for code blocks

scripts/tests/test_translation_fixer/test_code_blocks/data/en_doc.md

@@ -0,0 +1,44 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+```toml
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Mermaid diagram
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_gt.md

@@ -0,0 +1,45 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+```toml
+# Extra line
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_lines_number_lt.md

@@ -0,0 +1,45 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+The following block is missing first line:
+
+```toml
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_not_translated.md

@@ -0,0 +1,44 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+```toml
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_mermaid_translated.md

@@ -0,0 +1,44 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+```toml
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|требует| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_gt.md

@@ -0,0 +1,50 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+```toml
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+Extra code block
+
+```
+$ cd my_project
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_number_lt.md

@@ -0,0 +1,41 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+Missing code block...
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code.md

@@ -0,0 +1,46 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+The following block has wrong language code (should be TOML):
+
+```yaml
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/data/translated_doc_wrong_lang_code_2.md

@@ -0,0 +1,46 @@
+# Code blocks { #code-blocks }
+
+Some text
+
+```python
+# This is a sample Python code block
+def hello_world():
+    # Comment with indentation
+    print("Hello, world!")  # Print greeting
+```
+
+Some more text
+
+The following block has wrong language code (should be TOML):
+
+```
+# This is a sample TOML code block
+title = "TOML Example"  # Title of the document
+```
+
+And more text
+
+```console
+// Use the command "live" and pass the language code as a CLI argument
+$ python ./scripts/docs.py live es
+
+<span style="color: green;">[INFO]</span> Serving on http://127.0.0.1:8008
+<span style="color: green;">[INFO]</span> Start watching changes
+<span style="color: green;">[INFO]</span> Start detecting changes
+```
+
+And even more text
+
+```json
+{
+    // This is a sample JSON code block
+    "greeting": "Hello, world!" // Greeting
+}
+```
+
+Диаграма Mermaid
+
+```mermaid
+flowchart LR
+    stone(philosophers-stone) -->|requires| harry-1[harry v1]
+```

scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_lines_number_mismatch.py

@@ -0,0 +1,54 @@
+from pathlib import Path
+
+import pytest
+from typer.testing import CliRunner
+
+from scripts.translation_fixer import cli
+
+data_path = Path(
+    "scripts/tests/test_translation_fixer/test_code_blocks/data"
+).absolute()
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_lines_number_gt.md")],
+    indirect=True,
+)
+def test_gt(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_lines_number_gt.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Code block has different number of lines than the original block (5 vs 4)"
+    ) in result.output
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_lines_number_lt.md")],
+    indirect=True,
+)
+def test_lt(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    # assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_lines_number_lt.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Code block has different number of lines than the original block (3 vs 4)"
+    ) in result.output

scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_mermaid.py

@@ -0,0 +1,53 @@
+from pathlib import Path
+
+import pytest
+from typer.testing import CliRunner
+
+from scripts.translation_fixer import cli
+
+data_path = Path(
+    "scripts/tests/test_translation_fixer/test_code_blocks/data"
+).absolute()
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_mermaid_translated.md")],
+    indirect=True,
+)
+def test_translated(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 0, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_mermaid_translated.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert (
+        "Skipping mermaid code block replacement. This should be checked manually."
+    ) in result.output
+
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_mermaid_not_translated.md")],
+    indirect=True,
+)
+def test_not_translated(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 0, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_mermaid_not_translated.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert (
+        "Skipping mermaid code block replacement. This should be checked manually."
+    ) not in result.output

scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_number_mismatch.py

@@ -0,0 +1,56 @@
+from pathlib import Path
+
+import pytest
+from typer.testing import CliRunner
+
+from scripts.translation_fixer import cli
+
+data_path = Path(
+    "scripts/tests/test_translation_fixer/test_code_blocks/data"
+).absolute()
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_number_gt.md")],
+    indirect=True,
+)
+def test_gt(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_number_gt.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Number of code blocks does not match the number "
+        "in the original document (6 vs 5)"
+    ) in result.output
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_number_lt.md")],
+    indirect=True,
+)
+def test_lt(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    # assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_number_lt.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Number of code blocks does not match the number "
+        "in the original document (4 vs 5)"
+    ) in result.output

scripts/tests/test_translation_fixer/test_code_blocks/test_code_blocks_wrong_lang_code.py

@@ -0,0 +1,55 @@
+from pathlib import Path
+
+import pytest
+from typer.testing import CliRunner
+
+from scripts.translation_fixer import cli
+
+data_path = Path(
+    "scripts/tests/test_translation_fixer/test_code_blocks/data"
+).absolute()
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_wrong_lang_code.md")],
+    indirect=True,
+)
+def test_wrong_lang_code_1(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_wrong_lang_code.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Code block has different language than the original block ('yaml' vs 'toml')"
+    ) in result.output
+
+
+
+@pytest.mark.parametrize(
+    "copy_test_files",
+    [(f"{data_path}/en_doc.md", f"{data_path}/translated_doc_wrong_lang_code_2.md")],
+    indirect=True,
+)
+def test_wrong_lang_code_2(runner: CliRunner, root_dir: Path, copy_test_files):
+    result = runner.invoke(
+        cli,
+        ["fix-pages", "docs/lang/docs/doc.md"],
+    )
+    assert result.exit_code == 1, result.output
+
+    fixed_content = (root_dir / "docs" / "lang" / "docs" / "doc.md").read_text()
+    expected_content = Path(f"{data_path}/translated_doc_wrong_lang_code_2.md").read_text()
+
+    assert fixed_content == expected_content  # Translated doc remains unchanged
+    assert "Error processing docs/lang/docs/doc.md" in result.output
+    assert (
+        "Code block has different language than the original block ('' vs 'toml')"
+    ) in result.output

scripts/tests/test_translation_fixer/test_complex_doc/data/en_doc.md

@@ -103,6 +103,39 @@ def hello_world():// Print greeting
     print("Hello, world!")  //Print greeting without space after slashes
 ```
 
+## Code blocks with quadruple backticks { #code-blocks-with-quadruple-backticks }
+
+The following code block uses quadruple backticks.
+
+````python
+# Hello world function
+def hello_world():
+    print("Hello, world!")  # Print greeting
+````
+
+### Backticks number mismatch is fixable { #backticks-number-mismatch-is-fixable }
+
+The following code block has triple backticks in the original document, but quadruple backticks in the translated document.
+It will be fixed by the fixer tool (will convert to triple backticks).
+
+```Python
+# Some Python code
+```
+
+### Triple backticks inside quadruple backticks { #triple-backticks-inside-quadruple-backticks }
+
+Comments inside nested code block will NOT be preserved.
+
+````
+Here is a code block with quadruple backticks that contains triple backticks inside:
+
+```python
+# This is a sample Python code block
+def hello_world():
+    print("Hello, world!")  # Print greeting
+```
+
+````
 
 # Code includes { #code-includes }
 

scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc.md

@@ -101,6 +101,40 @@ def hello_world():// Печать приветствия
     print("Hello, world!")  //Печать приветствия без пробела после слешей
 ```
 
+## Блок кода с четырёхкратными обратными кавычками { #code-blocks-with-quadruple-backticks }
+
+Следующий блок кода содержит четырёхкратные обратные кавычки.
+
+````python
+# Функция приветствия
+def hello_world():
+    print("Hello, world")  # Печать приветствия
+````
+
+### Несоответствие обратных кавычек фиксится { #backticks-number-mismatch-is-fixable }
+
+Следующий блок кода имеет тройные обратные кавычки в оригинальном документе, но четырёхкратные обратные кавычки в переведённом документе.
+Это будет исправлено инструментом исправления (будет преобразовано в тройные обратные кавычки).
+
+````Python
+# Немного кода на Python
+````
+
+### Блок кода в тройных обратных кавычка внутри блока кода в четырёхкратных обратных кавычках { #triple-backticks-inside-quadruple-backticks }
+
+Комментарии внутри вложенного блока кода в тройных обратных кавычках НЕ БУДУТ сохранены.
+
+````
+Here is a code block with quadruple backticks that contains triple backticks inside:
+
+```python
+# Этот комментарий НЕ будет сохранён
+def hello_world():
+    print("Hello, world")  # Как и этот комментарий
+```
+
+````
+
 # Включения кода { #code-includes }
 
 ## Простые включения кода { #simple-code-includes }

scripts/tests/test_translation_fixer/test_complex_doc/data/translated_doc_expected.md

@@ -101,6 +101,40 @@ def hello_world():// Print greeting
     print("Hello, world!")  //Print greeting without space after slashes
 ```
 
+## Блок кода с четырёхкратными обратными кавычками { #code-blocks-with-quadruple-backticks }
+
+Следующий блок кода содержит четырёхкратные обратные кавычки.
+
+````python
+# Функция приветствия
+def hello_world():
+    print("Hello, world!")  # Печать приветствия
+````
+
+### Несоответствие обратных кавычек фиксится { #backticks-number-mismatch-is-fixable }
+
+Следующий блок кода имеет тройные обратные кавычки в оригинальном документе, но четырёхкратные обратные кавычки в переведённом документе.
+Это будет исправлено инструментом исправления (будет преобразовано в тройные обратные кавычки).
+
+```Python
+# Немного кода на Python
+```
+
+### Блок кода в тройных обратных кавычка внутри блока кода в четырёхкратных обратных кавычках { #triple-backticks-inside-quadruple-backticks }
+
+Комментарии внутри вложенного блока кода в тройных обратных кавычках НЕ БУДУТ сохранены.
+
+````
+Here is a code block with quadruple backticks that contains triple backticks inside:
+
+```python
+# This is a sample Python code block
+def hello_world():
+    print("Hello, world!")  # Print greeting
+```
+
+````
+
 # Включения кода { #code-includes }
 
 ## Простые включения кода { #simple-code-includes }