Skip to content

Commit 05d9aa3

Browse files
authored
Merge pull request #82 from eecs485staff/adds-stderr-printing-to-spec
Add Pipeline Debugging Tip: Print to stderr
2 parents f9f979c + 5593a3b commit 05d9aa3

File tree

1 file changed

+13
-6
lines changed

1 file changed

+13
-6
lines changed

README_Hadoop_Streaming.md

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -289,7 +289,14 @@ if __name__ == "__main__":
289289
## Tips and tricks
290290
These are some pro-tips for working with MapReduce programs written in Python for the Hadoop Streaming interface.
291291

292-
### Debugging
292+
### Print debug messages to `stderr`
293+
To avoid interfering with pipeline output in `stdout`, direct debugging print messages to `stderr`:
294+
295+
```python
296+
print("DEBUG finding bugs... ~(^._.)", file=sys.stderr)
297+
```
298+
299+
### Debugging with PDB
293300
We encounter a problem if we add a `breakpoint()` in `map.py`.
294301
```python
295302
for line in sys.stdin:
@@ -299,13 +306,13 @@ for line in sys.stdin:
299306
print(f"{word}\t1")
300307
```
301308

302-
PDB/PDB++ confuses the stdin being piped in from the input file for user input, so we get these errors:
309+
PDB confuses the stdin being piped in from the input file for user input, so we get these errors:
303310
```console
304311
$ cat input/input* | ./map.py
305312
...
306-
(Pdb++) *** SyntaxError: invalid syntax
307-
(Pdb++) *** SyntaxError: invalid syntax
308-
(Pdb++) *** SyntaxError: invalid syntax
313+
(Pdb) *** SyntaxError: invalid syntax
314+
(Pdb) *** SyntaxError: invalid syntax
315+
(Pdb) *** SyntaxError: invalid syntax
309316
...
310317
```
311318

@@ -325,7 +332,7 @@ Now our debugger works correctly.
325332
```console
326333
$ cat input/input* | ./map.py
327334
...
328-
(Pdb++)
335+
(Pdb)
329336
```
330337

331338
Don't forget to remove your temporary changes!

0 commit comments

Comments
 (0)