抽象批处理SFTP多目的地和自动重试
I do a lot of batch processing work using an in house Java framework. One of the activities that is particularly troublesome for us is SFTP where we run into two problems:
我使用内部Java框架进行了大量的批处理工作。对我们来说特别麻烦的活动之一是SFTP,我们遇到两个问题:
- Sending files to multiple destinations, where if one destination is unavailable the entire process stops.
- No automatic retries, where some network hiccup causes the batch process to stop.
将文件发送到多个目标,如果一个目标不可用,则整个过程停止。
没有自动重试,其中一些网络打嗝导致批处理停止。
Currently, we use Control-M for storing all of the SFTP related information:
目前,我们使用Control-M存储所有SFTP相关信息:
- Host & Path of From & To
- PGP key information
- Username & password
From&To的主持人和路径
PGP密钥信息
用户名密码
While it's certainly possible to do something with Control-M to solve these problems, I find it a cumbersome beast. I am looking for suggestions on a way of abstracting file transfer so that a batch process can generate a file, put it into some local folder, and decouple the transfer to the destination(s) in a robust fashion, whilst providing alerting/reporting when things go wrong without stopping.
虽然使用Control-M来解决这些问题当然是可行的,但我发现它是一个麻烦的野兽。我正在寻找关于抽象文件传输的方法的建议,以便批处理过程可以生成文件,将其放入某个本地文件夹,并以健壮的方式将传输分离到目标,同时提供警报/报告事情出问题而不停止。
In particular I am looking for anything that does this off the shelf as opposed to having to write such a tool myself.
特别是我正在寻找任何现成的东西,而不是自己编写这样的工具。
Environment
- Solaris 10
- Java 6
- Oracle 11
- Control-M 7
1 个解决方案
#1
0
- JScape SFTP is an enterprise-level commercial SFTP system.
- Aspera is an enterprise file transfer system which is much faster than SFTP; good for large files.
- Signiant is another enterprise file transfer and workflow system. It's similar in speed to Aspera.
JScape SFTP是一种企业级商用SFTP系统。
Aspera是一个企业文件传输系统,比SFTP快得多;适合大文件。
Signiant是另一种企业文件传输和工作流系统。它的速度与Aspera相似。