Need linux/python help with writing to csv file in cron

  • Context: Python 
  • Thread starter Thread starter devadmin
  • Start date Start date
  • Tags Tags
    Csv File Writing
Click For Summary

Discussion Overview

The discussion revolves around troubleshooting a Python script that writes to a CSV file when executed via cron. Participants explore potential issues related to script execution, file paths, and cron's behavior, while also sharing personal experiences with Python.

Discussion Character

  • Technical explanation
  • Debate/contested
  • Meta-discussion

Main Points Raised

  • One participant notes that the script works fine when run from the console but fails when executed via cron, suggesting a possible issue with the environment or execution context.
  • Another participant recommends checking system logs using commands like dmesg and cat /var/log/syslog to identify any errors, and suggests adding logging to the script to confirm it is running.
  • A participant realizes that the issue was due to using a relative path for the file, indicating a common pitfall when working with cron jobs.
  • One participant expresses frustration with cron's security settings, describing it as "extremely picky" and unhelpful in guiding users to correct configurations.
  • Several participants share their experiences with Python, discussing its growing popularity in data science and personal projects, as well as its clean syntax and extensive libraries.
  • Another participant mentions their preference for LabVIEW but appreciates Python for its optimization for developer time, particularly in database projects.
  • One participant comments on the initial challenge of nested indentation in Python but notes that using a good editor can alleviate this issue.

Areas of Agreement / Disagreement

Participants generally agree on the challenges of using cron and the importance of logging for troubleshooting. However, there is no consensus on the best practices for managing cron jobs or the relative merits of different programming languages.

Contextual Notes

Participants mention issues related to file paths and cron's execution environment, but do not resolve the broader implications of these challenges. The discussion includes personal anecdotes about programming experiences, which may not directly relate to the original troubleshooting issue.

Who May Find This Useful

Individuals interested in Python programming, troubleshooting cron jobs, or those exploring the use of Python in data science and database management may find this discussion relevant.

devadmin
I'm writing a simple script that writes to a csv file. When ran from console it works fine. When ran with cron, it doesn't write. No errors in the log. All files and folders are 777. The cron is active and running. What am I missing?
 
Technology news on Phys.org
What do:
Code: 
$ dmesg
$ cat /var/log/syslog
say?
If there was an error, it should show up in one of those.

How do you know that the script is actually running?

I recommend to start the script with something like
Code: 
#!/bin/bash
logger "Running simple script that writes to csv file"
so that there will always be an entry in the syslog. Note that standard output does not go anywhere.
If after that, there is still no entry in the log (after waiting for the script to trigger), that implies that the script did not actually run.
 
oh derp, I had a relative path to the file :D
 
I have found cron to be extremely picky about security settings, and not very helpful at steering you to set them correctly.
 
Seems like a good place to ask this, how many of you use Python at work or for personal projects? 3 years ago I had never once touched it now it’s my go to language. Still not my best but the dominance it has in data science plus the huge variety of tasks it can do are very appealing.
 
Jameson said:
Seems like a good place to ask this, how many of you use Python at work or for personal projects? 3 years ago I had never once touched it now it’s my go to language. Still not my best but the dominance it has in data science plus the huge variety of tasks it can do are very appealing.
Agreed, I'm just a few months in and I love it. It's so clean. Looking at nested indents can take getting used to. There are also so many libraries, great community.
 
The nested part was weird for me at first but with a good editor like Sublime the coloring and formatting help make it very obvious what code goes to what chunk.
 
I'm still slightly better at LabVIEW (and was therefore extremely excited when NI put out the Community Edition), but I like many things about Python. As text-based languages go, it's my favorite, because it appears optimized for developer time.

I do all my projects using various databases (either relational with SQL, or now I'm starting to use Neo4J with Cypher) and Python. That's it.
 

Similar threads

Python Writing modules that require supplementary files
  • · Replies 9 ·
Replies
9
Views
2K
Python [Python] Importing csv files and creating numpy arrays
  • · Replies 2 ·
Replies
2
Views
22K
Python How Can I Merge .dat Files into One Using Python?
  • · Replies 16 ·
Replies
16
Views
4K
Web Scraping Py Script Generates Error for Old Norse Chars
  • · Replies 4 ·
Replies
4
Views
2K
Python Running Electrum in Python from the CLI on Ubuntu 20
  • · Replies 2 ·
Replies
2
Views
2K
Could the reason why I can't select any kernels in VS Code be this error?
  • · Replies 7 ·
Replies
7
Views
3K
Python Best way to fix "externally managed environment" error?
  • · Replies 1 ·
Replies
1
Views
10K
Python Running CFD with python and bash: not in agreement
  • · Replies 21 ·
Replies
21
Views
3K
C/C++ Why overload prototype statement works in .h file but not in main file
  • · Replies 2 ·
Replies
2
Views
3K
Using crontab to clone private GitHub repo and run lualatex on Ubuntu
  • · Replies 16 ·
Replies
16
Views
4K